-- Logs begin at Wed 2022-08-10 16:15:33 UTC, end at Wed 2022-08-10 17:36:38 UTC. -- Aug 10 16:53:29.539686 dv-cinder-852724-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 10 16:53:29.546984 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 10 16:53:29.549030 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 10 16:53:29 2022] *** Aug 10 16:53:29.549030 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 10 16:53:29.549030 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 10 16:53:29.549030 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: nodename: dv-cinder-852724-3-iscsi Aug 10 16:53:29.549030 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: machine: x86_64 Aug 10 16:53:29.549030 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: clock source: unix Aug 10 16:53:29.549030 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: pcre jit disabled Aug 10 16:53:29.549030 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: detected number of CPU cores: 4 Aug 10 16:53:29.549030 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: current working directory: / Aug 10 16:53:29.549626 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: detected binary path: /usr/bin/uwsgi-core Aug 10 16:53:29.549626 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: your processes number limit is 47680 Aug 10 16:53:29.549626 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: your memory page size is 4096 bytes Aug 10 16:53:29.549626 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: detected max file descriptor number: 2048 Aug 10 16:53:29.549626 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: lock engine: pthread robust mutexes Aug 10 16:53:29.549626 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: thunder lock: enabled Aug 10 16:53:29.549626 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 10 16:53:29.549626 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 10 16:53:29.590984 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: Python main interpreter initialized at 0x564acf8db9c0 Aug 10 16:53:29.590984 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: python threads support enabled Aug 10 16:53:29.590984 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: your server socket listen backlog is limited to 100 connections Aug 10 16:53:29.590984 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: your mercy for graceful operations on workers is 90 seconds Aug 10 16:53:29.591359 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: mapped 403077 bytes (393 KB) for 2 cores Aug 10 16:53:29.591576 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: *** Operational MODE: preforking *** Aug 10 16:53:29.592457 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: *** uWSGI is running in multiple interpreter mode *** Aug 10 16:53:29.592457 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: spawned uWSGI master process (pid: 81598) Aug 10 16:53:29.592674 dv-cinder-852724-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 10 16:53:29.598386 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: spawned uWSGI worker 1 (pid: 81611, cores: 1) Aug 10 16:53:29.598386 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: spawned uWSGI worker 2 (pid: 81612, cores: 1) Aug 10 16:53:29.598386 dv-cinder-852724-3-iscsi devstack@keystone.service[81598]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 10 16:53:35.107669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 16:53:35.107669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: from cryptography.utils import int_from_bytes Aug 10 16:53:35.108618 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 16:53:35.108618 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: from cryptography.utils import int_from_bytes Aug 10 16:53:35.109070 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 16:53:35.109070 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: from cryptography.utils import int_from_bytes Aug 10 16:53:35.109207 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 16:53:35.109207 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: from cryptography.utils import int_from_bytes Aug 10 16:53:35.157570 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 10 16:53:35.157743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 10 16:53:35.157825 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 10 16:53:35.157904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 10 16:53:35.157971 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] command line args: [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 10 16:53:35.158030 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] command line args: [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 10 16:53:35.158204 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 10 16:53:35.158393 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 10 16:53:35.158490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ================================================================================ {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 10 16:53:35.158597 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ================================================================================ {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 10 16:53:35.158731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] admin_token = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.158874 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] admin_token = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.159038 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] config_dir = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.159123 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] config_dir = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.159268 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.159354 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.159465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] config_source = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.159543 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] config_source = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.159644 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] debug = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.159729 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] debug = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.159844 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.160153 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.160284 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.160398 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.160509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.160615 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.160720 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.160824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.160930 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.161029 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.161138 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.161237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_config_append = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.161341 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.161456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.161564 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_config_append = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.161669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_dir = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.161775 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.161886 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_file = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.161984 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_dir = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.162087 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.162195 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_file = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.162322 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.162486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.162572 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.162683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.162805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.162891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.163018 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.163104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.163243 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.163325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.163421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.163506 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.163588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.163707 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.163862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.164000 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.164145 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.164232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.164374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.164459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.164619 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.164704 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.164866 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.165000 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.165139 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.165249 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.165551 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.165639 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.45/identity {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.165789 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] publish_errors = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.165872 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.166027 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.166225 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.166310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.166460 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.166687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.166820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.45/identity {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.166923 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.167009 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] publish_errors = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.167159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] standard_threads = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.167298 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.167402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.167586 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.167673 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.167785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.167868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.167962 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] use_journal = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.168086 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.168164 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] use_json = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.168298 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] use_stderr = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.168382 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.168475 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] use_syslog = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.168660 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] standard_threads = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.168756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.168949 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.169040 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.169185 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.169263 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.170015 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.170107 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.170271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] use_journal = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.170450 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.170533 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] use_json = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.170629 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.170709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] use_stderr = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.170871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.170950 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] use_syslog = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.171121 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.171207 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 16:53:35.171360 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.171475 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.171609 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.171774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.171864 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.172094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.172178 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.172334 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.172420 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.172581 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.172758 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.172839 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.173054 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.173131 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.173302 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.173382 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.173539 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.173716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.173801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.174036 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.174119 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.174283 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.174464 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.174556 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.174800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.174909 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.175063 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.175254 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.175334 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.175507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.175585 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.175746 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.175825 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.175991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.176077 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.176222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.176300 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.176459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.176537 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.176710 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.176792 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.176959 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.177038 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.177190 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.177268 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.177422 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.177508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.177670 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.177749 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.177918 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.177997 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.178155 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.178240 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.178428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.178510 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.178660 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.178739 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.178888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.178966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.179118 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.179204 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.179344 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.179421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.179574 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.179653 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.179802 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.179880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.180031 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.180139 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.180248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.180347 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.180498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.180611 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.180744 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.180854 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.180963 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.181044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.181164 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.181242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.181368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.181453 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.181606 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.181687 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.181853 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.181943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.182083 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.182214 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.182302 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.182392 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.182554 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.182636 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.182783 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.182863 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.183017 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.183103 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.183254 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.183333 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.183482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.183561 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.183717 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.183803 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.183944 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.184029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.184178 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.184256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.184410 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.184499 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.184653 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.184732 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.184882 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.185031 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.185144 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.185230 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.185381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.185496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.185678 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.185752 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.185841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.185931 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.186078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.186158 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.186306 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.186454 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.186576 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.186673 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.186810 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.186889 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.187044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.187123 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.187273 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.187361 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.187530 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.187612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.187765 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.187847 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.187995 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.188149 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.188226 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.188298 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.188451 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.188537 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.188684 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.188771 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.188922 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.189001 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.189156 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.189235 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.189390 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.189482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.189623 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.189702 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.189858 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.189936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.190092 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.190170 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.190325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.190449 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.190586 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.190664 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.190821 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.190898 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.191056 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.191135 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.191288 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.191365 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.191517 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.191593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.191747 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.191833 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.191976 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.192061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.192206 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.192290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.192434 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.192513 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.192675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.192753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.192911 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.192988 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.193146 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.193226 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.193380 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.193457 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.193613 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.193691 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.193847 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.193928 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.194081 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.194160 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.194310 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.194407 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.external = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.194562 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.194643 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.194796 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.194914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.195028 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.195114 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.195256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.195337 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.password = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.195496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.195575 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.token = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.195731 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.195808 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.195969 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.196049 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.196205 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.196309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.196435 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.196524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.196684 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.196764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.196931 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.197011 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.197163 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.external = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.197243 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.197393 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.197472 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.197638 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.197718 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.197888 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.197968 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.198132 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.password = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.198210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.198396 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.token = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.198482 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.198618 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.198697 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.198853 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.198932 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.199099 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.199186 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.199329 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.199411 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.199574 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.199654 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.199822 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.199902 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.200053 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.200133 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.200287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.200373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.201035 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.201245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.201325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.201504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.201584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.201755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.201836 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.201995 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.202073 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.202228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.202307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.202461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.202544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.202721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.202802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.202978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.203057 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.203230 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.203313 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.203468 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.203547 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.203725 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.203805 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.203964 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.204046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.204215 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.204306 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.204453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.204531 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.204692 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.204772 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.204927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.205005 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.205171 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.205249 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.205416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.205497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.205646 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.205726 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.205882 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.205960 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.206114 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.206193 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.206382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.206465 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.206609 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.206688 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.206861 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.206943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.207096 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.207174 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.207336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.207414 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.207568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.207648 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.207801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.207880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.208044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.208122 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.208281 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.208362 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.208516 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.208596 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.208761 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.208841 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.208991 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.209072 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.209227 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.209307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.209474 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.209553 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.209709 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.209789 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.209942 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.210021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.210173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.210253 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.210426 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.210511 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.210665 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.210745 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.210907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.210997 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.211146 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.211229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.211395 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.211475 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.211628 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.211717 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.211876 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.211956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.212117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.212202 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.212353 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.212441 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.212579 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.212657 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.212815 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.212896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.213044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.213132 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.213278 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.213358 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.213504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.213582 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.213739 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.213826 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.213973 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.214061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.214201 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.214279 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.214469 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.214565 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.214707 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.214786 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.214942 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.215020 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.215188 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.215276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.215419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.215497 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.215650 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.215729 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.215883 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.215970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.216110 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.216842 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.216934 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.216996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.217143 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.217234 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.217382 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.217519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.217620 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.217757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.217857 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.217999 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.218106 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.218262 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.218343 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.218523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.218604 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.218773 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.218855 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.219023 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.219103 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.219271 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.219351 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.219506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.219586 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.219735 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.219813 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.219966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.220045 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.220196 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.220278 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.220431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.220509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.220663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.220739 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.220903 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.220983 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.221148 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.221226 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.221394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.221475 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.221638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.221717 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.221872 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.221951 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.222101 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.222182 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.222336 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.222465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.222593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.222675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.222826 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.222904 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.223054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.223134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.223286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.223363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.223532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.223611 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.223774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.223854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.224009 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.224087 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.224252 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.224330 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.224480 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.224561 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.224724 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.224802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.224958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.225037 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.225192 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.225272 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.225426 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.225505 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.225659 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.225738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.225887 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.225962 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.226117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.226198 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.226374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.226475 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.226618 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.226699 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.226845 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.226924 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.227089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.227177 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.227325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.227406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.227560 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.227640 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.227795 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.227883 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.228031 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.228110 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.228270 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.228349 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.228498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.228587 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.228733 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.228812 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.228967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] role.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.229045 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.229212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.229299 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] role.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.229443 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.229522 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.229680 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.229758 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.229926 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.230014 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.230172 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.230251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.230442 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.230524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.230690 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.230780 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.230925 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.231005 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.231167 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.231247 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.231403 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] role.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.231495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.231632 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.231711 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] role.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.231917 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.231999 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.232220 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.232398 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.232563 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.232638 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.232710 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.232870 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.232951 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.233167 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.233269 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.233411 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.233491 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.233658 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.233739 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.233905 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.233987 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.234141 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.234221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.234410 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.234494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.234658 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.234741 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.234901 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.235060 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.235140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.235322 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.235524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.235604 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.235759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.235839 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.236006 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.236096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.236239 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.236318 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.236495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.236575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.236754 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.236843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.237004 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.237084 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.237244 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.237323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.237483 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.237571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.237716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.237803 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.237953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.238033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.238191 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.238279 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.238445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.238527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.238698 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.238778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.238953 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.239047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.239201 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.239280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.239441 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.239526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.239662 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.239752 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.239892 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.239973 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.240127 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.240207 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.240364 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.240457 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.240604 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.240682 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.240838 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.240916 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.241085 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.241173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.241336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.241416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.241568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.241648 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.241811 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.241899 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.242056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.242136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.242304 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.242431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.connection = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.242565 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.242658 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.242804 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.242886 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.243042 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.243122 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.243288 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.243370 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.243525 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.243604 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.243754 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.243877 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.243988 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.244075 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.244224 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.244306 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.244469 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.244548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.connection = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.244706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.244796 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.244949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.245034 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.245184 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.245262 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.245422 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.245503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.245656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.245745 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.245892 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.246005 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.246121 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.246201 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.246369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.246463 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.246613 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.246694 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.246767 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 10 16:53:35.246839 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.247076 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.247311 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.247632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.247845 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.248200 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.248440 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.248682 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.248919 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 16:53:35.249088 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 10 16:53:35.276302 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.276532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.276758 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.277923 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.278153 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.278393 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.287072 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.288571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.315533 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.315706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.316336 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.316567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.368366 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.368618 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.419319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.419541 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.420215 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.420451 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.483606 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.483838 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.484056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.484146 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.484288 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.484374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.484518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.484669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.484800 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.484928 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.484928 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.485185 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.507535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.507692 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.507821 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.507946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.508072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.508210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.508337 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.508463 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.508588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.508723 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.508857 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.508982 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.509106 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.509230 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.520324 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.520489 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.533694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.533919 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.534135 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.534261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.534431 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.534572 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.534701 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.534825 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.534950 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.535073 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.535204 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.535336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.535336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.535497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 16:53:35.551786 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.551994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 16:53:35.552335 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.552536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 16:53:35.552814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.552938 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.553063 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 16:53:35.553194 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.553319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.553442 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.553568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 16:53:35.553769 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.553921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.554043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.554307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.554607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.565204 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.565482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.565752 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.566032 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.566465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.566749 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.567020 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.567302 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.573032 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.574372 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.576754 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.577034 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.577318 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.577604 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.577887 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.578114 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.578241 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.578427 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.578575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.578705 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.578829 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.578953 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.579078 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.579210 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.579337 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.579461 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.579586 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.579758 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.580034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.580306 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.580582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.580852 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.602931 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.605298 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.605652 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.605937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.606866 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.607180 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.610767 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.611045 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.611328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.612157 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.612447 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.612731 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.618007 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.618290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.619536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.619782 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.622390 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.623236 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.623525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.624094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.624094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.624381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.624655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.624927 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.625012 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.625304 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.625602 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.625896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.626176 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.626486 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.626767 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.627053 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.641436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.641717 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.643775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.644055 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.645716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.648058 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.648872 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.649169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.651485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.651782 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.653839 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.656237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.657043 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.659437 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.660226 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.660527 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.660814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.661107 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.661393 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.661672 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.662625 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.662927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.663216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.663502 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.663783 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.664066 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.677632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.677920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.678187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.678478 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.680093 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.680381 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.680650 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.680920 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.686156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.687760 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.688039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.688627 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.690183 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.690485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.690871 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.691154 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.691455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.691742 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.692021 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.693305 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.693893 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.693893 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.694171 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.694476 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.704164 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.704446 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.704728 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.705009 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.705287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.705571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.705847 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.706494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.706786 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.707069 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.707352 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.707631 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.707907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.708183 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.722497 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.724972 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.725460 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.727980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.728428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.729993 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.730942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.731520 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.731836 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.732126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.732872 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.734405 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.734701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.734995 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.738667 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.740270 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.741428 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.743067 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.743232 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.743524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.743811 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.744092 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.746027 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.746323 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.746641 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.746923 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.752706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.752998 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.753280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.755637 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.755930 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.756213 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.760274 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.760559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.760841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.761116 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.761389 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.761660 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.761935 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.762209 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.762618 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.762891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.763701 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.763792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 16:53:35.764023 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.764105 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.764307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.764392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.764585 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.764668 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.764859 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.765142 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.765421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.765691 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.766078 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.766370 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.766660 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.766934 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.767209 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.767473 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 16:53:35.798376 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x564acf8db9c0 pid: 81611 (default app) Aug 10 16:53:35.801289 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x564acf8db9c0 pid: 81612 (default app) Aug 10 16:53:35.801627 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60410269-c554-4bd1-93cc-1f9b0aa888c0 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:35.801913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60410269-c554-4bd1-93cc-1f9b0aa888c0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:35.802185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60410269-c554-4bd1-93cc-1f9b0aa888c0 None None] PATH_INFO: `/v3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:35.806461 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1/1] 10.222.0.45 () {50 vars in 791 bytes} [Wed Aug 10 16:53:35 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:53:44.617932 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddb7d9-cd02-4f98-af1c-3d611003e8d0 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:44.618574 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddb7d9-cd02-4f98-af1c-3d611003e8d0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:44.618574 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddb7d9-cd02-4f98-af1c-3d611003e8d0 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:44.623091 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1/2] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:53:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:53:44.628374 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64838f7a-9326-49fc-81c9-98bb5467f120 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:44.628778 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64838f7a-9326-49fc-81c9-98bb5467f120 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:44.628915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64838f7a-9326-49fc-81c9-98bb5467f120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:44.699724 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: INFO dbcounter [None req-64838f7a-9326-49fc-81c9-98bb5467f120 None None] Registered counter for database keystone Aug 10 16:53:44.719217 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG oslo_db.sqlalchemy.engines [None req-64838f7a-9326-49fc-81c9-98bb5467f120 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81611) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 10 16:53:44.734505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writer thread running {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 10 16:53:44.912763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG passlib.handlers.bcrypt [None req-64838f7a-9326-49fc-81c9-98bb5467f120 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81611) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 10 16:53:44.913094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG passlib.handlers.bcrypt [None req-64838f7a-9326-49fc-81c9-98bb5467f120 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81611) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 10 16:53:44.967385 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-64838f7a-9326-49fc-81c9-98bb5467f120 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:53:44.968971 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-64838f7a-9326-49fc-81c9-98bb5467f120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:53:45.101566 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 2/3] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:53:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 10 16:53:45.239378 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bddb7d9-cd02-4f98-af1c-3d611003e8d0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:45.310022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: INFO dbcounter [None req-0bddb7d9-cd02-4f98-af1c-3d611003e8d0 None None] Registered counter for database keystone Aug 10 16:53:45.331591 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG oslo_db.sqlalchemy.engines [None req-0bddb7d9-cd02-4f98-af1c-3d611003e8d0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81612) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 10 16:53:45.337846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writer thread running {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 10 16:53:45.457804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bddb7d9-cd02-4f98-af1c-3d611003e8d0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:45.462139 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba637862-fe6f-42b0-be32-c0699bc507fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:45.462806 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba637862-fe6f-42b0-be32-c0699bc507fa admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:45.463087 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba637862-fe6f-42b0-be32-c0699bc507fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:45.463358 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba637862-fe6f-42b0-be32-c0699bc507fa admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:45.472315 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba637862-fe6f-42b0-be32-c0699bc507fa admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:45.525797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ba637862-fe6f-42b0-be32-c0699bc507fa admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 10 16:53:45.526088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba637862-fe6f-42b0-be32-c0699bc507fa admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:45.535915 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 2/4] 10.222.0.45 () {58 vars in 1324 bytes} [Wed Aug 10 16:53:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 16:53:45.542068 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64838f7a-9326-49fc-81c9-98bb5467f120 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:45.556560 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64838f7a-9326-49fc-81c9-98bb5467f120 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:45.561629 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f6cf7-9cde-4370-a1ce-e6781571a75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:45.562744 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f6cf7-9cde-4370-a1ce-e6781571a75f admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:45.563028 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f6cf7-9cde-4370-a1ce-e6781571a75f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:45.563299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f6cf7-9cde-4370-a1ce-e6781571a75f admin admin] PATH_INFO: `/v3/projects/aff7d0faeed34f6193c691567931a8ff` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:45.564717 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f6cf7-9cde-4370-a1ce-e6781571a75f admin admin] RBAC: Authorizing `identity:get_project(project_id=aff7d0faeed34f6193c691567931a8ff)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:45.615788 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b93f6cf7-9cde-4370-a1ce-e6781571a75f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 10 16:53:45.616155 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f6cf7-9cde-4370-a1ce-e6781571a75f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:45.620793 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 3/5] 10.222.0.45 () {56 vars in 1207 bytes} [Wed Aug 10 16:53:45 2022] GET /identity/v3/projects/aff7d0faeed34f6193c691567931a8ff => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:53:47.597593 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149654c-b57e-488e-9820-9216216b5f37 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:47.598119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149654c-b57e-488e-9820-9216216b5f37 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:47.598119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149654c-b57e-488e-9820-9216216b5f37 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:47.602193 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 3/6] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:53:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:53:47.607808 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc148ee-d066-4c67-b719-45d48317e281 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:47.608242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc148ee-d066-4c67-b719-45d48317e281 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:47.608867 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc148ee-d066-4c67-b719-45d48317e281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:47.629862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2cc148ee-d066-4c67-b719-45d48317e281 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:53:47.631369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2cc148ee-d066-4c67-b719-45d48317e281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:53:47.639790 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 4/7] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:53:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 10 16:53:47.661914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149654c-b57e-488e-9820-9216216b5f37 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:47.674266 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149654c-b57e-488e-9820-9216216b5f37 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:47.678522 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb685e-aa0d-4679-876a-43f7c63f4b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:47.679116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb685e-aa0d-4679-876a-43f7c63f4b92 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:47.679395 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb685e-aa0d-4679-876a-43f7c63f4b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:47.679668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb685e-aa0d-4679-876a-43f7c63f4b92 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:47.687998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb685e-aa0d-4679-876a-43f7c63f4b92 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:47.689766 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb685e-aa0d-4679-876a-43f7c63f4b92 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:47.698601 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 4/8] 10.222.0.45 () {58 vars in 1324 bytes} [Wed Aug 10 16:53:47 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 16:53:47.705763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc148ee-d066-4c67-b719-45d48317e281 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:47.715111 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc148ee-d066-4c67-b719-45d48317e281 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:47.719659 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17abd97-7f72-45e7-9b1c-9df929f57650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:47.720275 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17abd97-7f72-45e7-9b1c-9df929f57650 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:47.720551 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17abd97-7f72-45e7-9b1c-9df929f57650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:47.720818 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17abd97-7f72-45e7-9b1c-9df929f57650 admin admin] PATH_INFO: `/v3/users/164faaac1ec04c50ba5358bf58d221cf` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:47.722106 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17abd97-7f72-45e7-9b1c-9df929f57650 admin admin] RBAC: Authorizing `identity:get_user(user_id=164faaac1ec04c50ba5358bf58d221cf)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:47.723956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17abd97-7f72-45e7-9b1c-9df929f57650 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:47.725451 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 5/9] 10.222.0.45 () {56 vars in 1198 bytes} [Wed Aug 10 16:53:47 2022] GET /identity/v3/users/164faaac1ec04c50ba5358bf58d221cf => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:53:49.800475 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4c8d2-8dc5-42b7-b838-2a5e1764204a None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:49.801012 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4c8d2-8dc5-42b7-b838-2a5e1764204a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:49.801012 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4c8d2-8dc5-42b7-b838-2a5e1764204a None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:49.802316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 5/10] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:53:49.807725 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22885e0a-bef6-4838-9087-684d22c84392 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:49.808123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22885e0a-bef6-4838-9087-684d22c84392 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:49.808282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22885e0a-bef6-4838-9087-684d22c84392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:49.826553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-22885e0a-bef6-4838-9087-684d22c84392 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:53:49.827722 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-22885e0a-bef6-4838-9087-684d22c84392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:53:49.835946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2ee73-3dd1-4dd9-8065-562e265e71c9 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:49.836222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2ee73-3dd1-4dd9-8065-562e265e71c9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:49.836493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2ee73-3dd1-4dd9-8065-562e265e71c9 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:49.837794 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 6/11] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:53:49.843341 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcaf1f-ce78-49c6-854c-12e84e0c2ea3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:49.843622 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcaf1f-ce78-49c6-854c-12e84e0c2ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:49.843921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcaf1f-ce78-49c6-854c-12e84e0c2ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:49.888556 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG passlib.handlers.bcrypt [None req-e3bcaf1f-ce78-49c6-854c-12e84e0c2ea3 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81612) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 10 16:53:49.888994 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG passlib.handlers.bcrypt [None req-e3bcaf1f-ce78-49c6-854c-12e84e0c2ea3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81612) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 10 16:53:49.909233 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 6/12] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:53:49.931576 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22885e0a-bef6-4838-9087-684d22c84392 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:49.934369 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e3bcaf1f-ce78-49c6-854c-12e84e0c2ea3 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:53:49.936701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e3bcaf1f-ce78-49c6-854c-12e84e0c2ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:53:49.955193 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:53:50.035959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22885e0a-bef6-4838-9087-684d22c84392 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:50.036177 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 7/13] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:53:50.058172 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcaf1f-ce78-49c6-854c-12e84e0c2ea3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:50.070235 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea690c6-5a33-4d74-8721-03296576032c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:50.070903 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea690c6-5a33-4d74-8721-03296576032c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:50.071178 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea690c6-5a33-4d74-8721-03296576032c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:50.071447 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea690c6-5a33-4d74-8721-03296576032c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:50.084022 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea690c6-5a33-4d74-8721-03296576032c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:53:50.093102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-eea690c6-5a33-4d74-8721-03296576032c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:53:50.093102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:53:50.093102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:53:50.093102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:53:50.093102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:53:50.093102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:53:50.093102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:53:50.093102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:53:50.093102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:53:50.093102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:53:50.093102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:53:50.093102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:53:50.093102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:53:50.093102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:53:50.093102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:53:50.093102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:53:50.094185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:53:50.094185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:53:50.094185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:53:50.094185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:53:50.094185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:53:50.094185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:53:50.094185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:53:50.094185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:53:50.094185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:53:50.094185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:53:50.094185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:53:50.094185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:53:50.094185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:53:50.094185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:53:50.094185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:53:50.094185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:53:50.095271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:53:50.095271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:53:50.095271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:53:50.095271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:53:50.095271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:53:50.095271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:53:50.095271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:53:50.095271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:53:50.095271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:53:50.095271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:53:50.095271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:53:50.095271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:53:50.095271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 7/14] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:53:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:53:50.100269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea690c6-5a33-4d74-8721-03296576032c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:50.162523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcaf1f-ce78-49c6-854c-12e84e0c2ea3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:50.164707 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:53:50.182603 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:53:50.198608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39dd587-71a0-4bac-8dfc-e2b03d0b6072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:50.199478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dd587-71a0-4bac-8dfc-e2b03d0b6072 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:50.199753 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dd587-71a0-4bac-8dfc-e2b03d0b6072 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:50.200136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dd587-71a0-4bac-8dfc-e2b03d0b6072 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:50.202106 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea690c6-5a33-4d74-8721-03296576032c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:50.235838 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e2ce4d-8a24-4e9a-8226-0621419edc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:50.236437 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2ce4d-8a24-4e9a-8226-0621419edc60 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:50.236719 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2ce4d-8a24-4e9a-8226-0621419edc60 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:50.236994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2ce4d-8a24-4e9a-8226-0621419edc60 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:50.237664 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e2ce4d-8a24-4e9a-8226-0621419edc60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:53:50.237966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e2ce4d-8a24-4e9a-8226-0621419edc60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:50.239539 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e2ce4d-8a24-4e9a-8226-0621419edc60 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:50.251894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 8/15] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:53:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:53:50.257543 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e2ce4d-8a24-4e9a-8226-0621419edc60 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:50.267808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39dd587-71a0-4bac-8dfc-e2b03d0b6072 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:50.269321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39dd587-71a0-4bac-8dfc-e2b03d0b6072 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:50.371851 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 8/16] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:53:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 16:53:50.378445 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39dd587-71a0-4bac-8dfc-e2b03d0b6072 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:50.381666 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:53:50.464828 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e2ce4d-8a24-4e9a-8226-0621419edc60 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:50.486595 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39dd587-71a0-4bac-8dfc-e2b03d0b6072 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:50.499848 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3384ef3-436d-418d-889b-c9b4e25afcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:50.500474 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3384ef3-436d-418d-889b-c9b4e25afcbe None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:50.500761 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3384ef3-436d-418d-889b-c9b4e25afcbe None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:50.501028 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3384ef3-436d-418d-889b-c9b4e25afcbe None admin] PATH_INFO: `/v3/users/164faaac1ec04c50ba5358bf58d221cf` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:50.502158 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3384ef3-436d-418d-889b-c9b4e25afcbe None admin] RBAC: Authorizing `identity:get_user(user_id=164faaac1ec04c50ba5358bf58d221cf)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:50.503654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3384ef3-436d-418d-889b-c9b4e25afcbe None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:50.505251 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 9/17] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:53:50 2022] GET /identity/v3/users/164faaac1ec04c50ba5358bf58d221cf => generated 247 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:53:50.510821 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3384ef3-436d-418d-889b-c9b4e25afcbe None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:50.521643 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21e223-03fa-434c-a34e-6de03010fcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:50.522596 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21e223-03fa-434c-a34e-6de03010fcbf None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:50.522879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21e223-03fa-434c-a34e-6de03010fcbf None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:50.523152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21e223-03fa-434c-a34e-6de03010fcbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:50.526189 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:53:50.541263 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21e223-03fa-434c-a34e-6de03010fcbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:50.545596 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21e223-03fa-434c-a34e-6de03010fcbf None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:50.561828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-4c21e223-03fa-434c-a34e-6de03010fcbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:53:50.561828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:53:50.561828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:53:50.561828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:53:50.561828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:53:50.561828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:53:50.561828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:53:50.561828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:53:50.561828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:53:50.561828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:53:50.561828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:53:50.561828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:53:50.561828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:53:50.561828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:53:50.561828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:53:50.561828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:53:50.562765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:53:50.562765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:53:50.562765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:53:50.562765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:53:50.562765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:53:50.562765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:53:50.562765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:53:50.562765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:53:50.562765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:53:50.562765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:53:50.562765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:53:50.562765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:53:50.562765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:53:50.562765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:53:50.562765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:53:50.562765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:53:50.562765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:53:50.563596 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:53:50.563596 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:53:50.563596 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:53:50.563596 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 9/18] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:53:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:53:50.568775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21e223-03fa-434c-a34e-6de03010fcbf None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:50.613773 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3384ef3-436d-418d-889b-c9b4e25afcbe None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:50.626828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:53:50.648051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4359ec56-4ad7-4591-898b-bf7b1187e777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:50.648682 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359ec56-4ad7-4591-898b-bf7b1187e777 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:50.648959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359ec56-4ad7-4591-898b-bf7b1187e777 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:50.649227 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359ec56-4ad7-4591-898b-bf7b1187e777 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:50.650631 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4359ec56-4ad7-4591-898b-bf7b1187e777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:50.652073 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4359ec56-4ad7-4591-898b-bf7b1187e777 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:50.654189 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 10/19] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:53:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:53:50.660075 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4359ec56-4ad7-4591-898b-bf7b1187e777 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:50.670982 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21e223-03fa-434c-a34e-6de03010fcbf None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:50.704108 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d81378-d2f3-4b1f-8a10-e9232b6743ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:50.704704 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d81378-d2f3-4b1f-8a10-e9232b6743ff None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:50.704995 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d81378-d2f3-4b1f-8a10-e9232b6743ff None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:50.705248 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d81378-d2f3-4b1f-8a10-e9232b6743ff None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:50.705885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d81378-d2f3-4b1f-8a10-e9232b6743ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:53:50.706183 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d81378-d2f3-4b1f-8a10-e9232b6743ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:50.707605 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d81378-d2f3-4b1f-8a10-e9232b6743ff None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:50.724877 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 10/20] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:53:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:53:50.741763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:53:50.762510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4359ec56-4ad7-4591-898b-bf7b1187e777 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:50.808796 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa85213-4ded-479a-b1a6-10975ffdab8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:50.809454 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa85213-4ded-479a-b1a6-10975ffdab8f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:50.809790 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa85213-4ded-479a-b1a6-10975ffdab8f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:50.810066 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa85213-4ded-479a-b1a6-10975ffdab8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:50.810845 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa85213-4ded-479a-b1a6-10975ffdab8f None admin] RBAC: Adding query filter params (role.id=7554b353873942fab506de830188aba0, scope.domain.id=default, user.id=164faaac1ec04c50ba5358bf58d221cf) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:53:50.811152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa85213-4ded-479a-b1a6-10975ffdab8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:50.812639 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa85213-4ded-479a-b1a6-10975ffdab8f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:50.820356 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 11/21] 10.222.0.45 () {56 vars in 1322 bytes} [Wed Aug 10 16:53:50 2022] GET /identity/v3/role_assignments?user.id=164faaac1ec04c50ba5358bf58d221cf&scope.domain.id=default&role.id=7554b353873942fab506de830188aba0 => generated 233 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:53:53.223031 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404a6b0-432c-4456-b287-9e8077976783 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:53.223944 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404a6b0-432c-4456-b287-9e8077976783 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:53.224421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404a6b0-432c-4456-b287-9e8077976783 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:53.226209 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 11/22] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:53:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:53:53.232121 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274f7a8-939b-47bb-bb53-55311e17baf0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:53.232979 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274f7a8-939b-47bb-bb53-55311e17baf0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:53.233432 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274f7a8-939b-47bb-bb53-55311e17baf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:53.263913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a274f7a8-939b-47bb-bb53-55311e17baf0 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:53:53.265505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a274f7a8-939b-47bb-bb53-55311e17baf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:53:53.358740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 12/23] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:53:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:53:53.383233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139c6bf-b27e-4211-a897-658e900d4345 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:53.383849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139c6bf-b27e-4211-a897-658e900d4345 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:53.384337 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139c6bf-b27e-4211-a897-658e900d4345 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:53.386174 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 12/24] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:53:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:53:53.391510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c95e8-c812-4ec4-a71a-b43e203308af None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:53.392085 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c95e8-c812-4ec4-a71a-b43e203308af None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:53.392579 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c95e8-c812-4ec4-a71a-b43e203308af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:53.395969 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139c6bf-b27e-4211-a897-658e900d4345 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:53.438381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a60c95e8-c812-4ec4-a71a-b43e203308af None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:53:53.438381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a60c95e8-c812-4ec4-a71a-b43e203308af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:53:53.548982 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 13/25] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:53:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:53:53.550656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e64cac-8fc6-46c8-b28e-54df5dff729a None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:53.551159 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e64cac-8fc6-46c8-b28e-54df5dff729a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:53.551639 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e64cac-8fc6-46c8-b28e-54df5dff729a None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:53.553575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 14/26] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:53:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:53:53.567883 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826489c-34e5-45b4-a08c-111e6f8f6e97 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:53.568243 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826489c-34e5-45b4-a08c-111e6f8f6e97 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:53.568439 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826489c-34e5-45b4-a08c-111e6f8f6e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:53.586422 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:53:53.612064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139c6bf-b27e-4211-a897-658e900d4345 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:53.615105 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8826489c-34e5-45b4-a08c-111e6f8f6e97 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:53:53.617637 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8826489c-34e5-45b4-a08c-111e6f8f6e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:53:53.634468 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:53:53.654380 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1394804-d6c5-45b6-92f5-b5e277ba77b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:53.655201 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1394804-d6c5-45b6-92f5-b5e277ba77b0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:53.655621 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1394804-d6c5-45b6-92f5-b5e277ba77b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:53.656101 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1394804-d6c5-45b6-92f5-b5e277ba77b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:53.668383 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1394804-d6c5-45b6-92f5-b5e277ba77b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:53:53.676267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-b1394804-d6c5-45b6-92f5-b5e277ba77b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:53:53.676267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:53:53.676267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:53:53.676267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:53:53.676267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:53:53.676267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:53:53.676267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:53:53.676267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:53:53.676267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:53:53.676267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:53:53.676267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:53:53.676267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:53:53.676267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:53:53.676267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:53:53.676267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:53:53.676267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:53:53.677307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:53:53.677307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:53:53.677307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:53:53.677307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:53:53.677307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:53:53.677307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:53:53.677307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:53:53.677307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:53:53.677307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:53:53.677307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:53:53.677307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:53:53.677307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:53:53.677307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:53:53.677307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:53:53.677307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:53:53.677307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:53:53.678218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:53:53.678218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:53:53.678218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:53:53.678218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:53:53.678218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:53:53.678218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:53:53.678218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:53:53.678218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:53:53.678218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:53:53.678218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:53:53.678218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:53:53.678218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:53:53.680124 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 13/27] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:53:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:53:53.681016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1394804-d6c5-45b6-92f5-b5e277ba77b0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:53.695173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 15/28] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:53:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:53:53.696434 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f7d8d-83f5-4bd3-b0c1-ab74934288c1 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:53.696720 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f7d8d-83f5-4bd3-b0c1-ab74934288c1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:53.696996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f7d8d-83f5-4bd3-b0c1-ab74934288c1 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:53.698706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 16/29] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:53:53.703158 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f7d8d-83f5-4bd3-b0c1-ab74934288c1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:53.794025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1394804-d6c5-45b6-92f5-b5e277ba77b0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:53.817651 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f7d8d-83f5-4bd3-b0c1-ab74934288c1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:53.859999 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e9310-e8bb-4dc7-b31c-3b4ee496d39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:53.860928 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e9310-e8bb-4dc7-b31c-3b4ee496d39e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:53.861359 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e9310-e8bb-4dc7-b31c-3b4ee496d39e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:53.861785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e9310-e8bb-4dc7-b31c-3b4ee496d39e None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:53.862677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e9310-e8bb-4dc7-b31c-3b4ee496d39e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:53:53.863155 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e9310-e8bb-4dc7-b31c-3b4ee496d39e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:53.864748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e9310-e8bb-4dc7-b31c-3b4ee496d39e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:53.866623 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:53:53.875480 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 17/30] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:53:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:53:53.877118 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d236a38-122b-4a20-b04b-1795249ed90f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:53.877556 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d236a38-122b-4a20-b04b-1795249ed90f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:53.883086 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d236a38-122b-4a20-b04b-1795249ed90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:53.898178 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8d236a38-122b-4a20-b04b-1795249ed90f None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:53:53.899645 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8d236a38-122b-4a20-b04b-1795249ed90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:53:53.947495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bac66-4803-47bf-a7c6-7b2d7246cb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:53.948532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bac66-4803-47bf-a7c6-7b2d7246cb96 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:53.949057 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bac66-4803-47bf-a7c6-7b2d7246cb96 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:53.949528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bac66-4803-47bf-a7c6-7b2d7246cb96 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:53.951176 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4bac66-4803-47bf-a7c6-7b2d7246cb96 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:53.952967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4bac66-4803-47bf-a7c6-7b2d7246cb96 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:53.973402 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 18/31] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:53:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:53:53.974237 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d236a38-122b-4a20-b04b-1795249ed90f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:54.092033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d236a38-122b-4a20-b04b-1795249ed90f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:54.121358 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:53:54.131239 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963004ef-2ac0-4ff5-95f6-12869c105813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:54.132113 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963004ef-2ac0-4ff5-95f6-12869c105813 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:54.132561 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963004ef-2ac0-4ff5-95f6-12869c105813 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:54.132976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963004ef-2ac0-4ff5-95f6-12869c105813 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:54.133772 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963004ef-2ac0-4ff5-95f6-12869c105813 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:54.135389 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963004ef-2ac0-4ff5-95f6-12869c105813 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:54.190377 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 14/32] 10.222.0.45 () {60 vars in 1131 bytes} [Wed Aug 10 16:53:53 2022] POST /identity/v3/roles => generated 219 bytes in 504 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:53:54.190377 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bac66-4803-47bf-a7c6-7b2d7246cb96 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:54.233227 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:53:54.311944 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bac66-4803-47bf-a7c6-7b2d7246cb96 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:54.366907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19deaec9-2b8d-40a5-8b54-6b3efc76b1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:54.367468 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19deaec9-2b8d-40a5-8b54-6b3efc76b1f6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:54.367748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19deaec9-2b8d-40a5-8b54-6b3efc76b1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:54.368019 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19deaec9-2b8d-40a5-8b54-6b3efc76b1f6 None admin] PATH_INFO: `/v3/users/164faaac1ec04c50ba5358bf58d221cf` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:54.369829 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19deaec9-2b8d-40a5-8b54-6b3efc76b1f6 None admin] RBAC: Authorizing `identity:get_user(user_id=164faaac1ec04c50ba5358bf58d221cf)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:54.371509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19deaec9-2b8d-40a5-8b54-6b3efc76b1f6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:54.378678 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 15/33] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:53:54 2022] GET /identity/v3/users/164faaac1ec04c50ba5358bf58d221cf => generated 247 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:53:54.379633 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19deaec9-2b8d-40a5-8b54-6b3efc76b1f6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:54.419871 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 19/34] 10.222.0.45 () {60 vars in 1131 bytes} [Wed Aug 10 16:53:53 2022] POST /identity/v3/roles => generated 225 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:53:54.421376 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f631517-6fea-425e-ade3-89892c6b7332 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:54.421664 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f631517-6fea-425e-ade3-89892c6b7332 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:54.421939 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f631517-6fea-425e-ade3-89892c6b7332 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:54.423529 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 20/35] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:53:54.424092 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f631517-6fea-425e-ade3-89892c6b7332 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:54.519614 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19deaec9-2b8d-40a5-8b54-6b3efc76b1f6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:54.550796 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:53:54.560127 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f631517-6fea-425e-ade3-89892c6b7332 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:54.560435 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b78e0-2637-4b9b-99d3-034051c4755b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:54.561020 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b78e0-2637-4b9b-99d3-034051c4755b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:54.561342 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b78e0-2637-4b9b-99d3-034051c4755b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:54.561617 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b78e0-2637-4b9b-99d3-034051c4755b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:54.576270 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46b78e0-2637-4b9b-99d3-034051c4755b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:54.578090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46b78e0-2637-4b9b-99d3-034051c4755b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:54.595960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-e46b78e0-2637-4b9b-99d3-034051c4755b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:53:54.595960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:53:54.595960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:53:54.595960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:53:54.595960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:53:54.595960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:53:54.595960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:53:54.595960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:53:54.595960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:53:54.595960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:53:54.595960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:53:54.595960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:53:54.595960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:53:54.595960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:53:54.595960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:53:54.595960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:53:54.596885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:53:54.596885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:53:54.596885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:53:54.596885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:53:54.596885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:53:54.596885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:53:54.596885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:53:54.596885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:53:54.596885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:53:54.596885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:53:54.596885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:53:54.596885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:53:54.596885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:53:54.596885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:53:54.596885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:53:54.596885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:53:54.596885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:53:54.597751 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:53:54.597751 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:53:54.597751 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:53:54.597751 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 16/36] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:53:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:53:54.598788 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeb1bc-0c9d-470f-b3a1-2b7d6d0a8729 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:54.598788 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeb1bc-0c9d-470f-b3a1-2b7d6d0a8729 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:54.598941 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeb1bc-0c9d-470f-b3a1-2b7d6d0a8729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:54.605599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2beda-a379-497d-bb80-e27441001a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:54.606257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2beda-a379-497d-bb80-e27441001a5b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:54.606590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2beda-a379-497d-bb80-e27441001a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:54.606867 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2beda-a379-497d-bb80-e27441001a5b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:54.608137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc2beda-a379-497d-bb80-e27441001a5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:54.609738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc2beda-a379-497d-bb80-e27441001a5b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:54.612823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 21/37] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:53:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:53:54.613386 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2beda-a379-497d-bb80-e27441001a5b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:54.626160 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5fbeb1bc-0c9d-470f-b3a1-2b7d6d0a8729 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:53:54.627987 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5fbeb1bc-0c9d-470f-b3a1-2b7d6d0a8729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:53:54.685324 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:53:54.714691 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 17/38] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:53:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:53:54.715408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbeb1bc-0c9d-470f-b3a1-2b7d6d0a8729 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:54.751936 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2beda-a379-497d-bb80-e27441001a5b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:54.793035 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be345af-42d1-4b2a-aa1c-3eb58452dfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:54.793737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be345af-42d1-4b2a-aa1c-3eb58452dfad None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:54.794757 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be345af-42d1-4b2a-aa1c-3eb58452dfad None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:54.795116 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be345af-42d1-4b2a-aa1c-3eb58452dfad None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:54.795804 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be345af-42d1-4b2a-aa1c-3eb58452dfad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:53:54.796102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be345af-42d1-4b2a-aa1c-3eb58452dfad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:54.797489 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be345af-42d1-4b2a-aa1c-3eb58452dfad None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:54.810289 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:53:54.815646 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 22/39] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:53:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:53:54.816457 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be345af-42d1-4b2a-aa1c-3eb58452dfad None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:54.828614 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbeb1bc-0c9d-470f-b3a1-2b7d6d0a8729 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:54.862338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9ec86-3b6f-4fbe-b1d5-a9291b8aba2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:54.863019 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9ec86-3b6f-4fbe-b1d5-a9291b8aba2f None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:54.863298 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9ec86-3b6f-4fbe-b1d5-a9291b8aba2f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:54.863575 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9ec86-3b6f-4fbe-b1d5-a9291b8aba2f None admin] PATH_INFO: `/v3/domains/default/users/164faaac1ec04c50ba5358bf58d221cf/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:54.865494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a9ec86-3b6f-4fbe-b1d5-a9291b8aba2f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=164faaac1ec04c50ba5358bf58d221cf, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:54.866987 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a9ec86-3b6f-4fbe-b1d5-a9291b8aba2f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:54.922745 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be345af-42d1-4b2a-aa1c-3eb58452dfad None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:54.939084 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:53:54.957982 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f1e38b-e269-48e8-bd91-4e1060ca8c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:54.958620 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1e38b-e269-48e8-bd91-4e1060ca8c9a None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:54.958900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1e38b-e269-48e8-bd91-4e1060ca8c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:54.959463 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1e38b-e269-48e8-bd91-4e1060ca8c9a None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:54.960122 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f1e38b-e269-48e8-bd91-4e1060ca8c9a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:54.961918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f1e38b-e269-48e8-bd91-4e1060ca8c9a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:54.965792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 18/40] 10.222.0.45 () {58 vars in 1361 bytes} [Wed Aug 10 16:53:54 2022] PUT /identity/v3/domains/default/users/164faaac1ec04c50ba5358bf58d221cf/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:53:54.967335 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0725c-7350-41ee-978e-dff3ae71703c None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:54.967621 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0725c-7350-41ee-978e-dff3ae71703c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:54.974581 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0725c-7350-41ee-978e-dff3ae71703c None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:54.974581 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 19/41] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:53:54.974581 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0725c-7350-41ee-978e-dff3ae71703c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:55.082389 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 23/42] 10.222.0.45 () {60 vars in 1131 bytes} [Wed Aug 10 16:53:54 2022] POST /identity/v3/roles => generated 223 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:53:55.082389 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f93cd-e3ba-4573-9827-ac7fd6bd82cd None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:55.082389 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f93cd-e3ba-4573-9827-ac7fd6bd82cd None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:55.082389 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f93cd-e3ba-4573-9827-ac7fd6bd82cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:55.106214 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-009f93cd-e3ba-4573-9827-ac7fd6bd82cd None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:53:55.108510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-009f93cd-e3ba-4573-9827-ac7fd6bd82cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:53:55.120270 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0725c-7350-41ee-978e-dff3ae71703c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:55.182557 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18304ab9-78b0-494c-b3c0-75cd2e21f3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:55.183232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18304ab9-78b0-494c-b3c0-75cd2e21f3aa None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:55.183517 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18304ab9-78b0-494c-b3c0-75cd2e21f3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:55.183792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18304ab9-78b0-494c-b3c0-75cd2e21f3aa None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:55.185220 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18304ab9-78b0-494c-b3c0-75cd2e21f3aa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:55.187106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18304ab9-78b0-494c-b3c0-75cd2e21f3aa None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:55.227439 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 24/43] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:53:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:53:55.251680 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f93cd-e3ba-4573-9827-ac7fd6bd82cd None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:55.285415 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:53:55.293436 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 20/44] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 10 16:53:54 2022] POST /identity/v3/projects => generated 301 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:53:55.395987 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f93cd-e3ba-4573-9827-ac7fd6bd82cd None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:55.399334 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:53:55.440018 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549d9f3-a9c9-4c18-9f5e-80d8a7e39549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:55.440625 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549d9f3-a9c9-4c18-9f5e-80d8a7e39549 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:55.440956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549d9f3-a9c9-4c18-9f5e-80d8a7e39549 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:55.441283 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549d9f3-a9c9-4c18-9f5e-80d8a7e39549 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:55.442578 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549d9f3-a9c9-4c18-9f5e-80d8a7e39549 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:55.444119 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549d9f3-a9c9-4c18-9f5e-80d8a7e39549 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:55.446154 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 25/45] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:53:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:53:55.452939 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18304ab9-78b0-494c-b3c0-75cd2e21f3aa None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:55.573989 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18304ab9-78b0-494c-b3c0-75cd2e21f3aa None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:55.610520 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b24fd-84ca-414a-8c6e-d8873bfa19fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:55.611164 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b24fd-84ca-414a-8c6e-d8873bfa19fc None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:55.611444 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b24fd-84ca-414a-8c6e-d8873bfa19fc None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:55.611719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b24fd-84ca-414a-8c6e-d8873bfa19fc None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:55.612407 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9b24fd-84ca-414a-8c6e-d8873bfa19fc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:55.613881 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9b24fd-84ca-414a-8c6e-d8873bfa19fc None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:55.721992 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 21/46] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 10 16:53:55 2022] POST /identity/v3/projects => generated 312 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:53:57.037657 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f378d-2aed-4440-a881-6a80bd7d32eb None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:57.038190 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f378d-2aed-4440-a881-6a80bd7d32eb None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:57.038190 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f378d-2aed-4440-a881-6a80bd7d32eb None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:57.039500 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 26/47] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:53:57.045221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ffad9-25e1-47d5-85a7-e9ead8b9a9da None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:57.045600 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ffad9-25e1-47d5-85a7-e9ead8b9a9da None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:57.045768 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ffad9-25e1-47d5-85a7-e9ead8b9a9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:57.063908 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-df8ffad9-25e1-47d5-85a7-e9ead8b9a9da None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:53:57.065089 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-df8ffad9-25e1-47d5-85a7-e9ead8b9a9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:53:57.109939 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:53:57.130321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 22/48] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:53:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:53:57.153347 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f378d-2aed-4440-a881-6a80bd7d32eb None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:57.254892 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f378d-2aed-4440-a881-6a80bd7d32eb None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:57.288133 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661bec36-df05-464d-b5f6-2de3a203a7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:57.288690 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661bec36-df05-464d-b5f6-2de3a203a7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:57.288971 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661bec36-df05-464d-b5f6-2de3a203a7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:57.289244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661bec36-df05-464d-b5f6-2de3a203a7b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:57.299645 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661bec36-df05-464d-b5f6-2de3a203a7b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:53:57.304319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-661bec36-df05-464d-b5f6-2de3a203a7b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:53:57.304319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:53:57.304319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:53:57.304319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:53:57.304319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:53:57.304319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:53:57.304319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:53:57.304319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:53:57.304319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:53:57.304319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:53:57.304319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:53:57.304319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:53:57.304319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:53:57.304319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:53:57.304319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:53:57.304319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:53:57.305247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:53:57.305247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:53:57.305247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:53:57.305247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:53:57.305247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:53:57.305247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:53:57.305247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:53:57.305247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:53:57.305247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:53:57.305247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:53:57.305247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:53:57.305247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:53:57.305247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:53:57.305247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:53:57.305247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:53:57.305247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:53:57.305997 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:53:57.305997 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:53:57.305997 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:53:57.305997 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:53:57.305997 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:53:57.305997 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:53:57.305997 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:53:57.305997 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:53:57.305997 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:53:57.305997 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:53:57.305997 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:53:57.305997 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:53:57.305997 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 27/49] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:53:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:53:57.312048 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ffad9-25e1-47d5-85a7-e9ead8b9a9da None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:57.411144 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ffad9-25e1-47d5-85a7-e9ead8b9a9da None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:57.444517 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3936e9-6370-43ba-a1ce-8ee9b31d8b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:57.445131 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3936e9-6370-43ba-a1ce-8ee9b31d8b4d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:57.445410 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3936e9-6370-43ba-a1ce-8ee9b31d8b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:57.445685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3936e9-6370-43ba-a1ce-8ee9b31d8b4d None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:57.446367 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3936e9-6370-43ba-a1ce-8ee9b31d8b4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:53:57.446680 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3936e9-6370-43ba-a1ce-8ee9b31d8b4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:57.448080 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3936e9-6370-43ba-a1ce-8ee9b31d8b4d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:57.459717 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 23/50] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:53:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:53:57.465144 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661bec36-df05-464d-b5f6-2de3a203a7b8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:57.495932 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:53:57.565200 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661bec36-df05-464d-b5f6-2de3a203a7b8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:57.598424 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d1f2f-f5c4-4c6f-8971-7fb15a85a9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:57.599047 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d1f2f-f5c4-4c6f-8971-7fb15a85a9bc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:57.599327 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d1f2f-f5c4-4c6f-8971-7fb15a85a9bc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:57.599598 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d1f2f-f5c4-4c6f-8971-7fb15a85a9bc None admin] PATH_INFO: `/v3/users/164faaac1ec04c50ba5358bf58d221cf` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:57.600719 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555d1f2f-f5c4-4c6f-8971-7fb15a85a9bc None admin] RBAC: Authorizing `identity:get_user(user_id=164faaac1ec04c50ba5358bf58d221cf)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:57.602176 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555d1f2f-f5c4-4c6f-8971-7fb15a85a9bc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:57.603966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 28/51] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:53:57 2022] GET /identity/v3/users/164faaac1ec04c50ba5358bf58d221cf => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:53:57.609413 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3936e9-6370-43ba-a1ce-8ee9b31d8b4d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:57.653734 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:53:57.709872 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3936e9-6370-43ba-a1ce-8ee9b31d8b4d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:57.742560 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855839e1-188d-4c97-9ac3-e5cc15c50c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:57.743180 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855839e1-188d-4c97-9ac3-e5cc15c50c59 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:57.743458 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855839e1-188d-4c97-9ac3-e5cc15c50c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:57.743724 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855839e1-188d-4c97-9ac3-e5cc15c50c59 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:57.744818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855839e1-188d-4c97-9ac3-e5cc15c50c59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:57.746249 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855839e1-188d-4c97-9ac3-e5cc15c50c59 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:57.748615 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 24/52] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:53:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:53:57.754246 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d1f2f-f5c4-4c6f-8971-7fb15a85a9bc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:57.766953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa2d84-a61b-44ff-be18-99be9c2df738 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:57.767242 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa2d84-a61b-44ff-be18-99be9c2df738 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:57.767520 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa2d84-a61b-44ff-be18-99be9c2df738 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:57.768815 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 25/53] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:53:57.773669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9032c-0208-4edd-9985-7be066edc2ec None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:57.773955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9032c-0208-4edd-9985-7be066edc2ec None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:57.774231 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9032c-0208-4edd-9985-7be066edc2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:57.792309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-57f9032c-0208-4edd-9985-7be066edc2ec None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:53:57.793484 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-57f9032c-0208-4edd-9985-7be066edc2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:53:57.851423 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:53:57.854329 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d1f2f-f5c4-4c6f-8971-7fb15a85a9bc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:57.859801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 26/54] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:53:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:53:57.883528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9032c-0208-4edd-9985-7be066edc2ec None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:57.887484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78539033-81ba-40ea-b6a3-1dc5789fb80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:57.888237 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78539033-81ba-40ea-b6a3-1dc5789fb80a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:57.888499 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78539033-81ba-40ea-b6a3-1dc5789fb80a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:57.888768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78539033-81ba-40ea-b6a3-1dc5789fb80a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:57.889484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78539033-81ba-40ea-b6a3-1dc5789fb80a None admin] RBAC: Adding query filter params (role.id=7554b353873942fab506de830188aba0, scope.domain.id=default, user.id=164faaac1ec04c50ba5358bf58d221cf) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:53:57.889788 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78539033-81ba-40ea-b6a3-1dc5789fb80a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:57.891272 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78539033-81ba-40ea-b6a3-1dc5789fb80a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:57.896866 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 29/55] 10.222.0.45 () {56 vars in 1322 bytes} [Wed Aug 10 16:53:57 2022] GET /identity/v3/role_assignments?user.id=164faaac1ec04c50ba5358bf58d221cf&scope.domain.id=default&role.id=7554b353873942fab506de830188aba0 => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:53:57.927037 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:53:57.987712 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9032c-0208-4edd-9985-7be066edc2ec None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:58.023114 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2556c0-0b0c-4019-9eb9-dbf28dc44b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:58.023750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2556c0-0b0c-4019-9eb9-dbf28dc44b55 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:58.024031 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2556c0-0b0c-4019-9eb9-dbf28dc44b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:58.024305 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2556c0-0b0c-4019-9eb9-dbf28dc44b55 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:58.025722 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2556c0-0b0c-4019-9eb9-dbf28dc44b55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:58.027227 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2556c0-0b0c-4019-9eb9-dbf28dc44b55 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:58.029251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 27/56] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:53:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:53:58.035320 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78539033-81ba-40ea-b6a3-1dc5789fb80a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:53:58.139088 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78539033-81ba-40ea-b6a3-1dc5789fb80a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:53:58.172756 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a6b22-932a-4118-a0fa-2803d8113c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:53:58.173380 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a6b22-932a-4118-a0fa-2803d8113c75 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:53:58.173659 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a6b22-932a-4118-a0fa-2803d8113c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:53:58.173933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a6b22-932a-4118-a0fa-2803d8113c75 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:53:58.174611 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a6b22-932a-4118-a0fa-2803d8113c75 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:53:58.176050 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a6b22-932a-4118-a0fa-2803d8113c75 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:53:58.263595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 30/57] 10.222.0.45 () {60 vars in 1140 bytes} [Wed Aug 10 16:53:58 2022] POST /identity/v3/projects => generated 298 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:00.260614 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878117a-0e21-4a50-856a-ca08992e3570 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:00.261123 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878117a-0e21-4a50-856a-ca08992e3570 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:00.261205 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878117a-0e21-4a50-856a-ca08992e3570 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:00.263038 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 28/58] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:00.268386 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1539263-c33b-4e94-a8a8-ea193d4077ef None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:00.268823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1539263-c33b-4e94-a8a8-ea193d4077ef None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:00.268922 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1539263-c33b-4e94-a8a8-ea193d4077ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:00.287525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e1539263-c33b-4e94-a8a8-ea193d4077ef None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:00.288698 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e1539263-c33b-4e94-a8a8-ea193d4077ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:00.299212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:00.352981 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 31/59] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:00.376974 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878117a-0e21-4a50-856a-ca08992e3570 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:00.475607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878117a-0e21-4a50-856a-ca08992e3570 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:00.484503 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:00.509008 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a735a85-457c-4068-b04a-8c22454d41d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:00.509622 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a735a85-457c-4068-b04a-8c22454d41d2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:00.509901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a735a85-457c-4068-b04a-8c22454d41d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:00.510174 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a735a85-457c-4068-b04a-8c22454d41d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:00.511359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a735a85-457c-4068-b04a-8c22454d41d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:00.513374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a735a85-457c-4068-b04a-8c22454d41d2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:00.514948 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 29/60] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:54:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:00.520514 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1539263-c33b-4e94-a8a8-ea193d4077ef None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:00.617882 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1539263-c33b-4e94-a8a8-ea193d4077ef None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:00.652464 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14891f42-520d-4f16-941a-0ad132ed0b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:00.653144 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14891f42-520d-4f16-941a-0ad132ed0b60 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:00.653516 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14891f42-520d-4f16-941a-0ad132ed0b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:00.653863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14891f42-520d-4f16-941a-0ad132ed0b60 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:00.654532 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14891f42-520d-4f16-941a-0ad132ed0b60 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:00.655982 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14891f42-520d-4f16-941a-0ad132ed0b60 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:00.742970 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 32/61] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 10 16:54:00 2022] POST /identity/v3/users => generated 275 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:03.786139 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34405f-3fd1-4df7-b3c2-083dd35f38f8 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:03.786724 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34405f-3fd1-4df7-b3c2-083dd35f38f8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:03.786798 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34405f-3fd1-4df7-b3c2-083dd35f38f8 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:03.788282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 30/62] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:03.794275 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20217c65-45ad-447a-8e0b-4a8eebf669b9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:03.794728 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20217c65-45ad-447a-8e0b-4a8eebf669b9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:03.794906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20217c65-45ad-447a-8e0b-4a8eebf669b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:03.805760 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:03.819341 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-20217c65-45ad-447a-8e0b-4a8eebf669b9 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:03.820542 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-20217c65-45ad-447a-8e0b-4a8eebf669b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:03.910931 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 33/63] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:03.944800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34405f-3fd1-4df7-b3c2-083dd35f38f8 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:04.092256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b42fd-0c69-4bc6-bc08-ff0f63e2a8d8 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:04.092834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34405f-3fd1-4df7-b3c2-083dd35f38f8 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:04.093117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b42fd-0c69-4bc6-bc08-ff0f63e2a8d8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:04.093587 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b42fd-0c69-4bc6-bc08-ff0f63e2a8d8 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:04.095157 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 34/64] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:04.103739 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41b902-2d7c-41a0-9b81-6f0c8a17ee61 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:04.104301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41b902-2d7c-41a0-9b81-6f0c8a17ee61 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:04.104791 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41b902-2d7c-41a0-9b81-6f0c8a17ee61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:04.125535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-cb41b902-2d7c-41a0-9b81-6f0c8a17ee61 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:04.127065 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cb41b902-2d7c-41a0-9b81-6f0c8a17ee61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:04.129741 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff4dcd-caaf-4559-9e77-f968652890ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:04.130396 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff4dcd-caaf-4559-9e77-f968652890ab None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:04.130849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff4dcd-caaf-4559-9e77-f968652890ab None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:04.131125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff4dcd-caaf-4559-9e77-f968652890ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:04.142827 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff4dcd-caaf-4559-9e77-f968652890ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:04.147762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d7ff4dcd-caaf-4559-9e77-f968652890ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:04.147762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:04.147762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:04.147762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:04.147762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:04.147762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:04.147762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:04.147762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:04.147762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:04.147762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:04.147762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:04.147762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:04.147762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:04.147762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:04.147762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:04.147762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:04.148623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:04.148623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:04.148623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:04.148623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:04.148623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:04.148623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:04.148623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:04.148623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:04.148623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:04.148623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:04.148623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:04.148623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:04.148623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:04.148623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:04.148623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:04.148623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:04.149599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:04.149599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:04.149599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:04.149599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:04.149599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:04.149599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:04.149599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:04.149599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:04.149599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:04.149599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:04.149599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:04.149599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:04.149599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 31/65] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:04.151604 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b13416-1820-4955-b076-07923dceb7d3 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:04.151895 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b13416-1820-4955-b076-07923dceb7d3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:04.152175 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b13416-1820-4955-b076-07923dceb7d3 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:04.153484 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 32/66] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:04.155094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b13416-1820-4955-b076-07923dceb7d3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:04.194720 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:04.214795 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 35/67] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:04.216357 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99629bff-eb08-48d0-9756-87e47fec21f2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:04.216775 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99629bff-eb08-48d0-9756-87e47fec21f2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:04.217176 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99629bff-eb08-48d0-9756-87e47fec21f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:04.247247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-99629bff-eb08-48d0-9756-87e47fec21f2 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:04.248707 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-99629bff-eb08-48d0-9756-87e47fec21f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:04.276962 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b13416-1820-4955-b076-07923dceb7d3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:04.314755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:04.314935 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2851a-8510-4ac5-899a-550a2624779c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:04.316158 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2851a-8510-4ac5-899a-550a2624779c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:04.316453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2851a-8510-4ac5-899a-550a2624779c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:04.316718 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2851a-8510-4ac5-899a-550a2624779c None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:04.317390 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f2851a-8510-4ac5-899a-550a2624779c None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:04.317691 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f2851a-8510-4ac5-899a-550a2624779c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:04.319116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f2851a-8510-4ac5-899a-550a2624779c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:04.327440 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 36/68] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:04.328512 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99629bff-eb08-48d0-9756-87e47fec21f2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:04.335250 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 33/69] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:04.341083 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2851a-8510-4ac5-899a-550a2624779c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:04.437576 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99629bff-eb08-48d0-9756-87e47fec21f2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:04.453565 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:04.457152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2851a-8510-4ac5-899a-550a2624779c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:04.476472 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbb557-1576-4dd4-9a29-66e572776c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:04.477400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbb557-1576-4dd4-9a29-66e572776c9f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:04.477886 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbb557-1576-4dd4-9a29-66e572776c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:04.478338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbb557-1576-4dd4-9a29-66e572776c9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:04.490953 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bbb557-1576-4dd4-9a29-66e572776c9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:04.492379 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1516e-934a-4c97-8b4c-5b7184e1d85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:04.493015 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1516e-934a-4c97-8b4c-5b7184e1d85e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:04.493293 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1516e-934a-4c97-8b4c-5b7184e1d85e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:04.493565 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1516e-934a-4c97-8b4c-5b7184e1d85e None admin] PATH_INFO: `/v3/users/b98eacd3b6474daabcffaf3ef31aea2b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:04.499276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-14bbb557-1576-4dd4-9a29-66e572776c9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:54:04.499276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:04.499276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:04.499276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:04.499276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:04.499276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:04.499276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:04.499276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:04.499276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:04.499276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:04.499276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:04.499276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:04.499276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:04.499276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:04.499276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:04.499276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:04.500123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:04.500123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:04.500123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:04.500123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:04.500123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:04.500123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:04.500123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:04.500123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:04.500123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:04.500123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:04.500123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:04.500123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:04.500123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:04.500123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:04.500123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:04.500123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:04.500885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:04.500885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:04.500885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:04.500885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:04.500885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:04.500885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:04.500885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:04.500885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:04.500885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:04.500885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:04.500885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:54:04.500885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:54:04.503309 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 37/70] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:54:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:04.504269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbb557-1576-4dd4-9a29-66e572776c9f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:04.511004 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b1516e-934a-4c97-8b4c-5b7184e1d85e None admin] RBAC: Authorizing `identity:get_user(user_id=b98eacd3b6474daabcffaf3ef31aea2b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:04.513027 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b1516e-934a-4c97-8b4c-5b7184e1d85e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:04.514973 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 34/71] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:04 2022] GET /identity/v3/users/b98eacd3b6474daabcffaf3ef31aea2b => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:04.515667 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1516e-934a-4c97-8b4c-5b7184e1d85e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:04.564158 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:04.616045 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbb557-1576-4dd4-9a29-66e572776c9f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:04.629853 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1516e-934a-4c97-8b4c-5b7184e1d85e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:04.652002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439fe5f-03ca-4694-af26-bfffa0630cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:04.652635 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439fe5f-03ca-4694-af26-bfffa0630cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:04.652915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439fe5f-03ca-4694-af26-bfffa0630cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:04.653837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439fe5f-03ca-4694-af26-bfffa0630cc0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:04.664236 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439fe5f-03ca-4694-af26-bfffa0630cc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:04.667350 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac22687-6a8f-44ce-8b29-d5f5268cebdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:04.667954 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac22687-6a8f-44ce-8b29-d5f5268cebdd None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:04.668235 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac22687-6a8f-44ce-8b29-d5f5268cebdd None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:04.668508 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac22687-6a8f-44ce-8b29-d5f5268cebdd None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:04.669010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-e439fe5f-03ca-4694-af26-bfffa0630cc0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:54:04.669010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:04.669010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:04.669010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:04.669010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:04.669010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:04.669010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:04.669010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:04.669010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:04.669010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:04.669010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:04.669010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:04.669010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:04.669010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:04.669010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:04.669010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:04.669975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:04.669975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:04.669975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:04.669975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:04.669975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:04.669975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:04.669975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:04.669975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:04.669975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:04.669975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:04.669975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:04.669975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:04.669975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:04.669975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:04.669975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:04.669975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:04.675900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:04.675900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:04.675900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:04.675900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:04.675900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:04.675900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:04.675900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:04.675900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:04.675900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:04.675900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:04.675900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:54:04.675900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:54:04.676612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac22687-6a8f-44ce-8b29-d5f5268cebdd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:04.676612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac22687-6a8f-44ce-8b29-d5f5268cebdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:04.676764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 38/72] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 10 16:54:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:04.676840 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac22687-6a8f-44ce-8b29-d5f5268cebdd None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:04.676916 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439fe5f-03ca-4694-af26-bfffa0630cc0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:04.684097 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 35/73] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:04.685405 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a35d6-88e6-4e0d-be5e-acca27fc422c None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:04.685648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a35d6-88e6-4e0d-be5e-acca27fc422c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:04.685918 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a35d6-88e6-4e0d-be5e-acca27fc422c None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:04.687152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 36/74] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:04.688156 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4111f-37e5-438d-9b5f-fa53787d79c3 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:04.688430 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4111f-37e5-438d-9b5f-fa53787d79c3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:04.688712 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4111f-37e5-438d-9b5f-fa53787d79c3 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:04.690103 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 37/75] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:04.694725 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4111f-37e5-438d-9b5f-fa53787d79c3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:04.714449 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:04.775539 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439fe5f-03ca-4694-af26-bfffa0630cc0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:04.788484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:04.797540 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4111f-37e5-438d-9b5f-fa53787d79c3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:04.809416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31edb1-e53b-4ef8-8aff-6a17afba9c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:04.810032 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31edb1-e53b-4ef8-8aff-6a17afba9c70 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:04.810341 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31edb1-e53b-4ef8-8aff-6a17afba9c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:04.810643 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31edb1-e53b-4ef8-8aff-6a17afba9c70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:04.831496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f1a97-8208-42f6-a4fa-1e0a6b50b631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:04.832094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f1a97-8208-42f6-a4fa-1e0a6b50b631 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:04.832382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f1a97-8208-42f6-a4fa-1e0a6b50b631 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:04.832654 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f1a97-8208-42f6-a4fa-1e0a6b50b631 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:04.833310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f1a97-8208-42f6-a4fa-1e0a6b50b631 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:04.833617 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f1a97-8208-42f6-a4fa-1e0a6b50b631 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:04.835033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f1a97-8208-42f6-a4fa-1e0a6b50b631 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:04.843434 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 38/76] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:54:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:04.844235 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f1a97-8208-42f6-a4fa-1e0a6b50b631 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:04.882315 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31edb1-e53b-4ef8-8aff-6a17afba9c70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:04.883825 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31edb1-e53b-4ef8-8aff-6a17afba9c70 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:04.925130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:04.943769 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f1a97-8208-42f6-a4fa-1e0a6b50b631 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:04.977281 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a818bce-1130-4a61-a269-e1df3683c39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:04.977832 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a818bce-1130-4a61-a269-e1df3683c39a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:04.978114 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a818bce-1130-4a61-a269-e1df3683c39a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:04.978414 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a818bce-1130-4a61-a269-e1df3683c39a None admin] PATH_INFO: `/v3/users/164faaac1ec04c50ba5358bf58d221cf` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:04.979550 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a818bce-1130-4a61-a269-e1df3683c39a None admin] RBAC: Authorizing `identity:get_user(user_id=164faaac1ec04c50ba5358bf58d221cf)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:04.981337 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a818bce-1130-4a61-a269-e1df3683c39a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:04.983288 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 39/77] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:04 2022] GET /identity/v3/users/164faaac1ec04c50ba5358bf58d221cf => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:04.984978 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55afc0-6f53-486c-ac27-9f577453b66d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:04.985288 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55afc0-6f53-486c-ac27-9f577453b66d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:04.985535 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55afc0-6f53-486c-ac27-9f577453b66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:04.993860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 39/78] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:54:04.993860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab726ff1-3225-4242-9c26-a9836a26b355 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:04.993860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab726ff1-3225-4242-9c26-a9836a26b355 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:04.993860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab726ff1-3225-4242-9c26-a9836a26b355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:05.003623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0b55afc0-6f53-486c-ac27-9f577453b66d None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:05.004825 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0b55afc0-6f53-486c-ac27-9f577453b66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:05.009630 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ab726ff1-3225-4242-9c26-a9836a26b355 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:05.010815 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ab726ff1-3225-4242-9c26-a9836a26b355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:05.019146 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:05.070187 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 40/79] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:05.071067 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55afc0-6f53-486c-ac27-9f577453b66d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:05.081342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 40/80] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:05.082176 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab726ff1-3225-4242-9c26-a9836a26b355 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:05.163246 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:05.184734 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55afc0-6f53-486c-ac27-9f577453b66d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:05.184949 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab726ff1-3225-4242-9c26-a9836a26b355 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:05.219581 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067503f-26a4-43db-947e-3030029d0e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:05.220205 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067503f-26a4-43db-947e-3030029d0e04 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:05.220303 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d35e5-542f-419a-b35f-d5cfc9c8a246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:05.220482 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067503f-26a4-43db-947e-3030029d0e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:05.220752 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067503f-26a4-43db-947e-3030029d0e04 None admin] PATH_INFO: `/v3/users/b98eacd3b6474daabcffaf3ef31aea2b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:05.220837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d35e5-542f-419a-b35f-d5cfc9c8a246 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:05.221082 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d35e5-542f-419a-b35f-d5cfc9c8a246 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:05.221351 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d35e5-542f-419a-b35f-d5cfc9c8a246 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:05.221879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067503f-26a4-43db-947e-3030029d0e04 None admin] RBAC: Authorizing `identity:get_user(user_id=b98eacd3b6474daabcffaf3ef31aea2b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:05.223354 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067503f-26a4-43db-947e-3030029d0e04 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:05.224934 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 41/81] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:05 2022] GET /identity/v3/users/b98eacd3b6474daabcffaf3ef31aea2b => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:05.229614 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067503f-26a4-43db-947e-3030029d0e04 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:05.234507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:05.289731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d35e5-542f-419a-b35f-d5cfc9c8a246 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:05.291240 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d35e5-542f-419a-b35f-d5cfc9c8a246 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:05.327523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067503f-26a4-43db-947e-3030029d0e04 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:05.365327 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2931e-7c1d-4b90-a849-cd0bccaddfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:05.366048 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2931e-7c1d-4b90-a849-cd0bccaddfa0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:05.366321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2931e-7c1d-4b90-a849-cd0bccaddfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:05.366625 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2931e-7c1d-4b90-a849-cd0bccaddfa0 None admin] PATH_INFO: `/v3/projects/1570ef44f3134332b1fc668345a37479` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:05.367771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2931e-7c1d-4b90-a849-cd0bccaddfa0 None admin] RBAC: Authorizing `identity:get_project(project_id=1570ef44f3134332b1fc668345a37479)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:05.369231 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2931e-7c1d-4b90-a849-cd0bccaddfa0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:05.370884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 42/82] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:05 2022] GET /identity/v3/projects/1570ef44f3134332b1fc668345a37479 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:05.371649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2931e-7c1d-4b90-a849-cd0bccaddfa0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:05.387359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:05.397231 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 41/83] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:54:05.398134 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d35e5-542f-419a-b35f-d5cfc9c8a246 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:05.441244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:05.474632 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2931e-7c1d-4b90-a849-cd0bccaddfa0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:05.498924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d35e5-542f-419a-b35f-d5cfc9c8a246 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:05.508033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3dd2f1-1c1c-422e-9685-f093c98bcde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:05.508671 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dd2f1-1c1c-422e-9685-f093c98bcde1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:05.508946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dd2f1-1c1c-422e-9685-f093c98bcde1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:05.509218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dd2f1-1c1c-422e-9685-f093c98bcde1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:05.510317 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3dd2f1-1c1c-422e-9685-f093c98bcde1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:05.511797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3dd2f1-1c1c-422e-9685-f093c98bcde1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:05.514375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 43/84] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:54:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:05.515151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3dd2f1-1c1c-422e-9685-f093c98bcde1 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:05.534775 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842a8e1-883d-480e-ae33-4192e3d28fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:05.535404 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842a8e1-883d-480e-ae33-4192e3d28fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:05.535681 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842a8e1-883d-480e-ae33-4192e3d28fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:05.535951 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842a8e1-883d-480e-ae33-4192e3d28fb7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:05.546007 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842a8e1-883d-480e-ae33-4192e3d28fb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:05.550833 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-f842a8e1-883d-480e-ae33-4192e3d28fb7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:05.550833 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:05.550833 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:05.550833 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:05.550833 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:05.550833 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:05.550833 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:05.550833 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:05.550833 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:05.550833 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:05.550833 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:05.550833 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:05.550833 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:05.550833 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:05.550833 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:05.550833 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:05.551655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:05.551655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:05.551655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:05.551655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:05.551655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:05.551655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:05.551655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:05.551655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:05.551655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:05.551655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:05.551655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:05.551655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:05.551655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:05.551655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:05.551655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:05.551655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:05.552445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:05.552445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:05.552445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:05.552445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:05.552445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:05.552445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:05.552445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:05.552445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:05.552445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:05.552445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:05.552445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:05.552445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:54:05.552445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 42/85] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:05.553776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842a8e1-883d-480e-ae33-4192e3d28fb7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:05.597578 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:05.617684 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3dd2f1-1c1c-422e-9685-f093c98bcde1 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:05.655456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e37166-878b-4928-80c7-2b713059acce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:05.656072 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e37166-878b-4928-80c7-2b713059acce None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:05.656305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842a8e1-883d-480e-ae33-4192e3d28fb7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:05.656406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e37166-878b-4928-80c7-2b713059acce None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:05.656626 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e37166-878b-4928-80c7-2b713059acce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:05.657870 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:05.692034 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffc67a-75cc-4e94-832d-0bc8facad96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:05.692638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffc67a-75cc-4e94-832d-0bc8facad96d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:05.692923 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffc67a-75cc-4e94-832d-0bc8facad96d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:05.693198 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffc67a-75cc-4e94-832d-0bc8facad96d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:05.693920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ffc67a-75cc-4e94-832d-0bc8facad96d None admin] RBAC: Adding query filter params (role.id=dfb72dc1f3744c8e957abcc4ac967dac, scope.project.id=1570ef44f3134332b1fc668345a37479, user.id=b98eacd3b6474daabcffaf3ef31aea2b) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:05.694230 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ffc67a-75cc-4e94-832d-0bc8facad96d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:05.695727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ffc67a-75cc-4e94-832d-0bc8facad96d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:05.701041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 43/86] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:54:05 2022] GET /identity/v3/role_assignments?user.id=b98eacd3b6474daabcffaf3ef31aea2b&scope.project.id=1570ef44f3134332b1fc668345a37479&role.id=dfb72dc1f3744c8e957abcc4ac967dac => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:05.701822 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffc67a-75cc-4e94-832d-0bc8facad96d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:05.742412 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e37166-878b-4928-80c7-2b713059acce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:05.743924 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e37166-878b-4928-80c7-2b713059acce None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:05.814498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffc67a-75cc-4e94-832d-0bc8facad96d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:05.835926 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:05.852628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b9d3b-d801-4b45-af38-8052640f5f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:05.853641 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b9d3b-d801-4b45-af38-8052640f5f17 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:05.853937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b9d3b-d801-4b45-af38-8052640f5f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:05.854211 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b9d3b-d801-4b45-af38-8052640f5f17 None admin] PATH_INFO: `/v3/projects/1570ef44f3134332b1fc668345a37479` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:05.855417 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b9d3b-d801-4b45-af38-8052640f5f17 None admin] RBAC: Authorizing `identity:get_project(project_id=1570ef44f3134332b1fc668345a37479)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:05.857189 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b9d3b-d801-4b45-af38-8052640f5f17 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:05.862799 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 44/87] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:54:05.862799 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e37166-878b-4928-80c7-2b713059acce None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:05.863037 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 44/88] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:05 2022] GET /identity/v3/projects/1570ef44f3134332b1fc668345a37479 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:05.863037 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b9d3b-d801-4b45-af38-8052640f5f17 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:05.900559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:05.969645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e37166-878b-4928-80c7-2b713059acce None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:05.970088 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b9d3b-d801-4b45-af38-8052640f5f17 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:06.006572 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd210e4-c13c-4fe6-bf3f-7e3a9a9007a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:06.007179 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd210e4-c13c-4fe6-bf3f-7e3a9a9007a9 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:06.007456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd210e4-c13c-4fe6-bf3f-7e3a9a9007a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:06.007724 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd210e4-c13c-4fe6-bf3f-7e3a9a9007a9 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:06.008363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd210e4-c13c-4fe6-bf3f-7e3a9a9007a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:06.008868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51da3d3-5627-423e-a78e-135d752bf284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:06.009479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51da3d3-5627-423e-a78e-135d752bf284 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:06.009761 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51da3d3-5627-423e-a78e-135d752bf284 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:06.009852 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd210e4-c13c-4fe6-bf3f-7e3a9a9007a9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:06.010041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51da3d3-5627-423e-a78e-135d752bf284 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:06.010754 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51da3d3-5627-423e-a78e-135d752bf284 None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:06.011058 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51da3d3-5627-423e-a78e-135d752bf284 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:06.012574 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51da3d3-5627-423e-a78e-135d752bf284 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:06.026167 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 45/89] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:06.026167 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51da3d3-5627-423e-a78e-135d752bf284 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:06.117709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:06.123988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51da3d3-5627-423e-a78e-135d752bf284 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:06.156939 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e6ba6-ef59-4c88-825d-0b304f649381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:06.157569 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e6ba6-ef59-4c88-825d-0b304f649381 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:06.157840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e6ba6-ef59-4c88-825d-0b304f649381 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:06.158104 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e6ba6-ef59-4c88-825d-0b304f649381 None admin] PATH_INFO: `/v3/projects/1570ef44f3134332b1fc668345a37479` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:06.159263 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e6ba6-ef59-4c88-825d-0b304f649381 None admin] RBAC: Authorizing `identity:get_project(project_id=1570ef44f3134332b1fc668345a37479)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:06.160709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e6ba6-ef59-4c88-825d-0b304f649381 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:06.162832 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 46/90] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:06 2022] GET /identity/v3/projects/1570ef44f3134332b1fc668345a37479 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:06.167725 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e6ba6-ef59-4c88-825d-0b304f649381 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:06.173979 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 45/91] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 10 16:54:05 2022] POST /identity/v3/users => generated 289 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:06.184170 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd210e4-c13c-4fe6-bf3f-7e3a9a9007a9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:06.240279 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:06.317918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e6ba6-ef59-4c88-825d-0b304f649381 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:06.319472 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd210e4-c13c-4fe6-bf3f-7e3a9a9007a9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:06.362018 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6eb54-84e7-450e-b015-4085bacd9ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:06.362701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6eb54-84e7-450e-b015-4085bacd9ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:06.362978 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6eb54-84e7-450e-b015-4085bacd9ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:06.363248 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6eb54-84e7-450e-b015-4085bacd9ef6 None admin] PATH_INFO: `/v3/users/b98eacd3b6474daabcffaf3ef31aea2b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:06.364423 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6eb54-84e7-450e-b015-4085bacd9ef6 None admin] RBAC: Authorizing `identity:get_user(user_id=b98eacd3b6474daabcffaf3ef31aea2b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:06.365420 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ba942b-c2c7-47b8-a76d-7af54aed9049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:06.365887 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6eb54-84e7-450e-b015-4085bacd9ef6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:06.366045 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba942b-c2c7-47b8-a76d-7af54aed9049 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:06.366343 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba942b-c2c7-47b8-a76d-7af54aed9049 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:06.366658 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba942b-c2c7-47b8-a76d-7af54aed9049 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:06.367818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 46/92] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:06 2022] GET /identity/v3/users/b98eacd3b6474daabcffaf3ef31aea2b => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:06.367964 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ba942b-c2c7-47b8-a76d-7af54aed9049 None admin] RBAC: Adding query filter params (role.id=7554b353873942fab506de830188aba0, scope.project.id=1570ef44f3134332b1fc668345a37479, user.id=164faaac1ec04c50ba5358bf58d221cf) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:06.368223 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ba942b-c2c7-47b8-a76d-7af54aed9049 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:06.368315 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6eb54-84e7-450e-b015-4085bacd9ef6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:06.369718 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ba942b-c2c7-47b8-a76d-7af54aed9049 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:06.378842 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 47/93] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:54:06 2022] GET /identity/v3/role_assignments?user.id=164faaac1ec04c50ba5358bf58d221cf&scope.project.id=1570ef44f3134332b1fc668345a37479&role.id=7554b353873942fab506de830188aba0 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:06.379352 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ba942b-c2c7-47b8-a76d-7af54aed9049 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:06.415844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:06.535904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6eb54-84e7-450e-b015-4085bacd9ef6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:06.547124 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:06.547312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ba942b-c2c7-47b8-a76d-7af54aed9049 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:06.582876 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ccf0c1-c6a3-4550-ad28-020d7d7f9127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:06.583494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccf0c1-c6a3-4550-ad28-020d7d7f9127 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:06.583832 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccf0c1-c6a3-4550-ad28-020d7d7f9127 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:06.584103 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccf0c1-c6a3-4550-ad28-020d7d7f9127 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:06.584923 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ccf0c1-c6a3-4550-ad28-020d7d7f9127 None admin] RBAC: Adding query filter params (role.id=a3e5816caacc425c9a6dd381fe0eddf0, scope.project.id=1570ef44f3134332b1fc668345a37479, user.id=b98eacd3b6474daabcffaf3ef31aea2b) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:06.585099 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe8fb8-97b5-4f61-8c5c-d042aa5899b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:06.585300 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ccf0c1-c6a3-4550-ad28-020d7d7f9127 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:06.585579 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe8fb8-97b5-4f61-8c5c-d042aa5899b7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:06.585863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe8fb8-97b5-4f61-8c5c-d042aa5899b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:06.586141 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe8fb8-97b5-4f61-8c5c-d042aa5899b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:06.586768 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ccf0c1-c6a3-4550-ad28-020d7d7f9127 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:06.597033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 47/94] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:54:06 2022] GET /identity/v3/role_assignments?user.id=b98eacd3b6474daabcffaf3ef31aea2b&scope.project.id=1570ef44f3134332b1fc668345a37479&role.id=a3e5816caacc425c9a6dd381fe0eddf0 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:06.666147 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe8fb8-97b5-4f61-8c5c-d042aa5899b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:06.667594 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe8fb8-97b5-4f61-8c5c-d042aa5899b7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:06.698979 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:06.780543 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 48/95] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:54:06.786137 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ccf0c1-c6a3-4550-ad28-020d7d7f9127 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:06.918059 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ccf0c1-c6a3-4550-ad28-020d7d7f9127 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:06.961856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd84eb-c09c-4b7d-afe7-053b353eae6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:06.962560 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd84eb-c09c-4b7d-afe7-053b353eae6f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:06.962841 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd84eb-c09c-4b7d-afe7-053b353eae6f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:06.963112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd84eb-c09c-4b7d-afe7-053b353eae6f None admin] PATH_INFO: `/v3/projects/29443ed08048475986823123d96ec005` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:06.964254 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd84eb-c09c-4b7d-afe7-053b353eae6f None admin] RBAC: Authorizing `identity:get_project(project_id=29443ed08048475986823123d96ec005)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:06.966392 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd84eb-c09c-4b7d-afe7-053b353eae6f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:06.968179 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 48/96] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:06 2022] GET /identity/v3/projects/29443ed08048475986823123d96ec005 => generated 312 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:06.974622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe8fb8-97b5-4f61-8c5c-d042aa5899b7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:07.099465 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe8fb8-97b5-4f61-8c5c-d042aa5899b7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:07.134924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adbdd30-b220-4149-894b-ade65a8c8b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:07.135877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbdd30-b220-4149-894b-ade65a8c8b59 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:07.136380 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbdd30-b220-4149-894b-ade65a8c8b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:07.136867 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbdd30-b220-4149-894b-ade65a8c8b59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:07.138368 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adbdd30-b220-4149-894b-ade65a8c8b59 None admin] RBAC: Adding query filter params (role.id=dfb72dc1f3744c8e957abcc4ac967dac, scope.project.id=29443ed08048475986823123d96ec005, user.id=b98eacd3b6474daabcffaf3ef31aea2b) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:07.142382 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:07.142835 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adbdd30-b220-4149-894b-ade65a8c8b59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:07.144561 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adbdd30-b220-4149-894b-ade65a8c8b59 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:07.150925 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 49/97] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:54:06 2022] GET /identity/v3/role_assignments?user.id=b98eacd3b6474daabcffaf3ef31aea2b&scope.project.id=29443ed08048475986823123d96ec005&role.id=dfb72dc1f3744c8e957abcc4ac967dac => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:08.246925 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35ab1b-12c4-4132-895b-9c3ddf6babf2 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:08.247544 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35ab1b-12c4-4132-895b-9c3ddf6babf2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:08.247544 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35ab1b-12c4-4132-895b-9c3ddf6babf2 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:08.248811 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 49/98] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:08.254209 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab7693-4b81-4673-9c84-30aed352ae19 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:08.254633 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab7693-4b81-4673-9c84-30aed352ae19 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:08.254819 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab7693-4b81-4673-9c84-30aed352ae19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:08.279470 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5eab7693-4b81-4673-9c84-30aed352ae19 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:08.280663 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5eab7693-4b81-4673-9c84-30aed352ae19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:08.375834 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 50/99] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:08.411126 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35ab1b-12c4-4132-895b-9c3ddf6babf2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:08.468918 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:08.544190 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35ab1b-12c4-4132-895b-9c3ddf6babf2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:08.594656 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554ba4a5-f06f-4e7d-9bea-ef66ae5f3ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:08.595293 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ba4a5-f06f-4e7d-9bea-ef66ae5f3ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:08.595573 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ba4a5-f06f-4e7d-9bea-ef66ae5f3ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:08.595860 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ba4a5-f06f-4e7d-9bea-ef66ae5f3ae6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:08.611891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554ba4a5-f06f-4e7d-9bea-ef66ae5f3ae6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:08.616648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-554ba4a5-f06f-4e7d-9bea-ef66ae5f3ae6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:08.616648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:08.616648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:08.616648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:08.616648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:08.616648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:08.616648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:08.616648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:08.616648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:08.616648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:08.616648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:08.616648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:08.616648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:08.616648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:08.616648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:08.616648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:08.617528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:08.617528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:08.617528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:08.617528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:08.617528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:08.617528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:08.617528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:08.617528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:08.617528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:08.617528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:08.617528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:08.617528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:08.617528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:08.617528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:08.617528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:08.617528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:08.618310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:08.618310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:08.618310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:08.618310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:08.618310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:08.618310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:08.618310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:08.618310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:08.618310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:08.618310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:08.618310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:08.618310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:08.618310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 50/100] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:08.631564 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab7693-4b81-4673-9c84-30aed352ae19 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:08.753062 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab7693-4b81-4673-9c84-30aed352ae19 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:08.783033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:08.791724 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ccd4ef-5dde-4f06-a3c1-81d8cdc9af67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:08.792339 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ccd4ef-5dde-4f06-a3c1-81d8cdc9af67 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:08.792621 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ccd4ef-5dde-4f06-a3c1-81d8cdc9af67 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:08.792894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ccd4ef-5dde-4f06-a3c1-81d8cdc9af67 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:08.793622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ccd4ef-5dde-4f06-a3c1-81d8cdc9af67 None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:08.793927 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ccd4ef-5dde-4f06-a3c1-81d8cdc9af67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:08.795425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ccd4ef-5dde-4f06-a3c1-81d8cdc9af67 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:08.807991 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 51/101] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:08.814077 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554ba4a5-f06f-4e7d-9bea-ef66ae5f3ae6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:08.959831 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:08.964192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554ba4a5-f06f-4e7d-9bea-ef66ae5f3ae6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:09.008539 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77e4c2-cee0-4190-a1d4-d95e5bf22e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:09.009225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77e4c2-cee0-4190-a1d4-d95e5bf22e95 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:09.009514 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77e4c2-cee0-4190-a1d4-d95e5bf22e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:09.009782 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77e4c2-cee0-4190-a1d4-d95e5bf22e95 None admin] PATH_INFO: `/v3/users/b98eacd3b6474daabcffaf3ef31aea2b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:09.014033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77e4c2-cee0-4190-a1d4-d95e5bf22e95 None admin] RBAC: Authorizing `identity:get_user(user_id=b98eacd3b6474daabcffaf3ef31aea2b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:09.015545 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77e4c2-cee0-4190-a1d4-d95e5bf22e95 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:09.017340 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 51/102] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:08 2022] GET /identity/v3/users/b98eacd3b6474daabcffaf3ef31aea2b => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:09.022937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ccd4ef-5dde-4f06-a3c1-81d8cdc9af67 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:09.110963 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac26e5-4284-4b29-87c6-a29016e1566f None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:09.111286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac26e5-4284-4b29-87c6-a29016e1566f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:09.111520 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac26e5-4284-4b29-87c6-a29016e1566f None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:09.112876 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 52/103] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:54:09.119315 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a85da-55b4-4dfb-b738-f8eaaac70434 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:09.119597 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a85da-55b4-4dfb-b738-f8eaaac70434 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:09.119874 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a85da-55b4-4dfb-b738-f8eaaac70434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:09.143788 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7f3a85da-55b4-4dfb-b738-f8eaaac70434 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:09.145211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7f3a85da-55b4-4dfb-b738-f8eaaac70434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:09.174728 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ccd4ef-5dde-4f06-a3c1-81d8cdc9af67 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:09.236937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb225d56-2ed9-4df4-80d6-196f893da46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:09.237574 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb225d56-2ed9-4df4-80d6-196f893da46c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:09.237885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb225d56-2ed9-4df4-80d6-196f893da46c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:09.238170 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb225d56-2ed9-4df4-80d6-196f893da46c None admin] PATH_INFO: `/v3/projects/1570ef44f3134332b1fc668345a37479` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:09.240180 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb225d56-2ed9-4df4-80d6-196f893da46c None admin] RBAC: Authorizing `identity:get_project(project_id=1570ef44f3134332b1fc668345a37479)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:09.241638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb225d56-2ed9-4df4-80d6-196f893da46c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:09.243479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 52/104] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:09 2022] GET /identity/v3/projects/1570ef44f3134332b1fc668345a37479 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:09.249558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb225d56-2ed9-4df4-80d6-196f893da46c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:09.271342 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 53/105] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:54:09.304274 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a85da-55b4-4dfb-b738-f8eaaac70434 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:09.312956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:09.366470 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb225d56-2ed9-4df4-80d6-196f893da46c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:09.469596 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5a320-e407-49c0-a96a-cd8697513fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:09.470270 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5a320-e407-49c0-a96a-cd8697513fb9 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:09.470574 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5a320-e407-49c0-a96a-cd8697513fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:09.470842 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5a320-e407-49c0-a96a-cd8697513fb9 None admin] PATH_INFO: `/v3/projects/1570ef44f3134332b1fc668345a37479/users/b98eacd3b6474daabcffaf3ef31aea2b/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:09.475700 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d5a320-e407-49c0-a96a-cd8697513fb9 None admin] RBAC: Authorizing `identity:create_grant(project_id=1570ef44f3134332b1fc668345a37479, user_id=b98eacd3b6474daabcffaf3ef31aea2b, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:09.477151 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d5a320-e407-49c0-a96a-cd8697513fb9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:09.506666 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:09.510341 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a85da-55b4-4dfb-b738-f8eaaac70434 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:09.557850 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f94b81-edfa-4df1-addc-f8b20da1d85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:09.558513 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f94b81-edfa-4df1-addc-f8b20da1d85f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:09.558796 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f94b81-edfa-4df1-addc-f8b20da1d85f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:09.559067 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f94b81-edfa-4df1-addc-f8b20da1d85f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:09.572405 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f94b81-edfa-4df1-addc-f8b20da1d85f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:09.577277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d8f94b81-edfa-4df1-addc-f8b20da1d85f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:54:09.577277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:09.577277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:09.577277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:09.577277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:09.577277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:09.577277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:09.577277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:09.577277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:09.577277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:09.577277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:09.577277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:09.577277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:09.577277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:09.577277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:09.577277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:09.578191 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:09.578191 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:09.578191 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:09.578191 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:09.578191 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:09.578191 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:09.578191 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:09.578191 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:09.578191 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:09.578191 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:09.578191 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:09.578191 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:09.578191 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:09.578191 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:09.578191 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:09.578191 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:09.579016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:09.579016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:09.579016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:09.579016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:09.579016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:09.579016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:09.579016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:09.579016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:09.579016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:09.579016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:09.579016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:54:09.579016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:09.580313 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 54/106] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:54:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:09.583928 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f998ce-9519-4c60-b7e4-38c55eafa5c1 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:09.584211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f998ce-9519-4c60-b7e4-38c55eafa5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:09.584483 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f998ce-9519-4c60-b7e4-38c55eafa5c1 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:09.585959 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 55/107] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:09.589493 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 53/108] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 10 16:54:09 2022] PUT /identity/v3/projects/1570ef44f3134332b1fc668345a37479/users/b98eacd3b6474daabcffaf3ef31aea2b/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:54:09.590245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f998ce-9519-4c60-b7e4-38c55eafa5c1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:09.622450 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25e301-8081-434b-8ef6-d511f4bf2d6d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:09.622450 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25e301-8081-434b-8ef6-d511f4bf2d6d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:09.622450 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25e301-8081-434b-8ef6-d511f4bf2d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:09.655483 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5c25e301-8081-434b-8ef6-d511f4bf2d6d None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:09.656718 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5c25e301-8081-434b-8ef6-d511f4bf2d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:09.745494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f998ce-9519-4c60-b7e4-38c55eafa5c1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:09.794162 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 54/109] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:09.795429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bdfe6-ea87-4146-87a7-1098f7df5050 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:09.795717 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bdfe6-ea87-4146-87a7-1098f7df5050 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:09.795991 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bdfe6-ea87-4146-87a7-1098f7df5050 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:09.797430 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 55/110] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:09.803618 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379f757-7d11-4fd1-9774-4b22577842e7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:09.803901 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379f757-7d11-4fd1-9774-4b22577842e7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:09.804176 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379f757-7d11-4fd1-9774-4b22577842e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:09.822179 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ffbcf-3320-4fe3-bdf0-1f4614c5d31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:09.822873 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ffbcf-3320-4fe3-bdf0-1f4614c5d31d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:09.823207 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ffbcf-3320-4fe3-bdf0-1f4614c5d31d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:09.823483 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ffbcf-3320-4fe3-bdf0-1f4614c5d31d None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:09.824192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ffbcf-3320-4fe3-bdf0-1f4614c5d31d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:09.824511 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ffbcf-3320-4fe3-bdf0-1f4614c5d31d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:09.826048 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ffbcf-3320-4fe3-bdf0-1f4614c5d31d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:09.838416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f379f757-7d11-4fd1-9774-4b22577842e7 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:09.838416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f379f757-7d11-4fd1-9774-4b22577842e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:09.846309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 56/111] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:09.848469 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ffbcf-3320-4fe3-bdf0-1f4614c5d31d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:09.873720 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:09.878003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:09.912749 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 56/112] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:09.919855 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379f757-7d11-4fd1-9774-4b22577842e7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:09.960790 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ffbcf-3320-4fe3-bdf0-1f4614c5d31d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:10.022824 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf170736-787f-462f-a8a1-778ac77c1e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:10.023464 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf170736-787f-462f-a8a1-778ac77c1e42 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:10.023743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf170736-787f-462f-a8a1-778ac77c1e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:10.024015 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf170736-787f-462f-a8a1-778ac77c1e42 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:10.050914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf170736-787f-462f-a8a1-778ac77c1e42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:10.062151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-cf170736-787f-462f-a8a1-778ac77c1e42 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:54:10.062151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:10.062151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:10.062151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:10.062151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:10.062151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:10.062151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:10.062151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:10.062151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:10.062151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:10.062151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:10.062151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:10.062151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:10.062151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:10.062151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:10.062151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:10.063071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:10.063071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:10.063071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:10.063071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:10.063071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:10.063071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:10.063071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:10.063071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:10.063071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:10.063071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:10.063071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:10.063071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:10.063071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:10.063071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:10.063071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:10.063071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:10.063822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:10.063822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:10.063822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:10.063822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:10.063822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:10.063822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:10.063822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:10.063822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:10.063822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:10.063822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:10.063822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:54:10.063822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:10.064917 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 57/113] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 10 16:54:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:10.065715 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf170736-787f-462f-a8a1-778ac77c1e42 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:10.089356 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379f757-7d11-4fd1-9774-4b22577842e7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:10.137178 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba001be-3174-4bbf-90e6-e0c32e835bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:10.138463 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba001be-3174-4bbf-90e6-e0c32e835bfe None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:10.138873 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba001be-3174-4bbf-90e6-e0c32e835bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:10.139151 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba001be-3174-4bbf-90e6-e0c32e835bfe None admin] PATH_INFO: `/v3/users/164faaac1ec04c50ba5358bf58d221cf` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:10.143496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba001be-3174-4bbf-90e6-e0c32e835bfe None admin] RBAC: Authorizing `identity:get_user(user_id=164faaac1ec04c50ba5358bf58d221cf)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:10.145329 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba001be-3174-4bbf-90e6-e0c32e835bfe None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:10.147777 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 57/114] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:09 2022] GET /identity/v3/users/164faaac1ec04c50ba5358bf58d221cf => generated 247 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:10.148547 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba001be-3174-4bbf-90e6-e0c32e835bfe None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:10.228051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:10.240314 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:10.271015 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf170736-787f-462f-a8a1-778ac77c1e42 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:10.303072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba001be-3174-4bbf-90e6-e0c32e835bfe None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:10.315778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d787e-068e-4e63-8b66-1bb93fac816e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:10.316471 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d787e-068e-4e63-8b66-1bb93fac816e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:10.316749 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d787e-068e-4e63-8b66-1bb93fac816e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:10.317030 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d787e-068e-4e63-8b66-1bb93fac816e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:10.332106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d787e-068e-4e63-8b66-1bb93fac816e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:10.338459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-169d787e-068e-4e63-8b66-1bb93fac816e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:54:10.338459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:10.338459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:10.338459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:10.338459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:10.338459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:10.338459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:10.338459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:10.338459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:10.338459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:10.338459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:10.338459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:10.338459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:10.338459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:10.338459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:10.338459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:10.339350 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:10.339350 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:10.339350 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:10.339350 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:10.339350 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:10.339350 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:10.339350 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:10.339350 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:10.339350 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:10.339350 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:10.339350 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:10.339350 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:10.339350 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:10.339350 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:10.339350 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:10.339350 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:10.340117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:10.340117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:10.340117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:10.340117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:10.340117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:10.340117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:10.340117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:10.340117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:10.340117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:10.340117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:10.340117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:54:10.340117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:10.340117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 58/115] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:10.346328 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d787e-068e-4e63-8b66-1bb93fac816e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:10.352336 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c64dc-a060-4dd1-bce5-dd5e58c4296c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:10.352958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c64dc-a060-4dd1-bce5-dd5e58c4296c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:10.353232 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c64dc-a060-4dd1-bce5-dd5e58c4296c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:10.353501 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c64dc-a060-4dd1-bce5-dd5e58c4296c None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:10.354169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c64dc-a060-4dd1-bce5-dd5e58c4296c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:10.354498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c64dc-a060-4dd1-bce5-dd5e58c4296c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:10.355885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c64dc-a060-4dd1-bce5-dd5e58c4296c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:10.364362 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 58/116] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:54:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:10.365560 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5fd2a-13b0-4e34-adaf-ea585996ce37 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:10.365841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5fd2a-13b0-4e34-adaf-ea585996ce37 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:10.366110 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5fd2a-13b0-4e34-adaf-ea585996ce37 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:10.367514 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 59/117] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:10.368042 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5fd2a-13b0-4e34-adaf-ea585996ce37 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:10.461683 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d787e-068e-4e63-8b66-1bb93fac816e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:10.476710 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5fd2a-13b0-4e34-adaf-ea585996ce37 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:10.478378 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:10.497452 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c7ac9-aa5c-443c-9402-63239c45dd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:10.500878 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c7ac9-aa5c-443c-9402-63239c45dd98 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:10.500878 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c7ac9-aa5c-443c-9402-63239c45dd98 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:10.500878 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c7ac9-aa5c-443c-9402-63239c45dd98 None admin] PATH_INFO: `/v3/projects/1570ef44f3134332b1fc668345a37479` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:10.500878 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05c7ac9-aa5c-443c-9402-63239c45dd98 None admin] RBAC: Authorizing `identity:get_project(project_id=1570ef44f3134332b1fc668345a37479)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:10.501251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05c7ac9-aa5c-443c-9402-63239c45dd98 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:10.503052 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 59/118] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:10 2022] GET /identity/v3/projects/1570ef44f3134332b1fc668345a37479 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:10.503993 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba345cf-de60-476e-bc41-9a3cd4505e36 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:10.504270 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba345cf-de60-476e-bc41-9a3cd4505e36 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:10.504547 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba345cf-de60-476e-bc41-9a3cd4505e36 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:10.505730 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 60/119] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:10.510412 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba345cf-de60-476e-bc41-9a3cd4505e36 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:10.513806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a99b50-c62c-4e20-9574-66f70f2aed89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:10.514461 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a99b50-c62c-4e20-9574-66f70f2aed89 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:10.514750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a99b50-c62c-4e20-9574-66f70f2aed89 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:10.515027 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a99b50-c62c-4e20-9574-66f70f2aed89 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:10.515710 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a99b50-c62c-4e20-9574-66f70f2aed89 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:10.515997 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a99b50-c62c-4e20-9574-66f70f2aed89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:10.517362 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a99b50-c62c-4e20-9574-66f70f2aed89 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:10.521454 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:10.532162 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 60/120] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:10.532162 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0edd1-f42e-4822-879d-010a112f83d0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:10.532162 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0edd1-f42e-4822-879d-010a112f83d0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:10.532162 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0edd1-f42e-4822-879d-010a112f83d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:10.547912 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-dde0edd1-f42e-4822-879d-010a112f83d0 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:10.549095 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-dde0edd1-f42e-4822-879d-010a112f83d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:10.633411 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 61/121] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:10.633411 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0edd1-f42e-4822-879d-010a112f83d0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:10.635243 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba345cf-de60-476e-bc41-9a3cd4505e36 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:10.673211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5931818-20a9-4b95-82d7-746eb59b62d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:10.673837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5931818-20a9-4b95-82d7-746eb59b62d7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:10.674114 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5931818-20a9-4b95-82d7-746eb59b62d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:10.674415 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5931818-20a9-4b95-82d7-746eb59b62d7 None admin] PATH_INFO: `/v3/users/b98eacd3b6474daabcffaf3ef31aea2b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:10.675531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5931818-20a9-4b95-82d7-746eb59b62d7 None admin] RBAC: Authorizing `identity:get_user(user_id=b98eacd3b6474daabcffaf3ef31aea2b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:10.676990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5931818-20a9-4b95-82d7-746eb59b62d7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:10.678589 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 61/122] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:10 2022] GET /identity/v3/users/b98eacd3b6474daabcffaf3ef31aea2b => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:10.679723 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8d097-a3c7-44ce-b037-2a807b1f8158 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:10.680028 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8d097-a3c7-44ce-b037-2a807b1f8158 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:10.680319 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8d097-a3c7-44ce-b037-2a807b1f8158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:10.697411 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d1f8d097-a3c7-44ce-b037-2a807b1f8158 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:10.698602 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d1f8d097-a3c7-44ce-b037-2a807b1f8158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:10.736638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:10.743924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0edd1-f42e-4822-879d-010a112f83d0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:10.768877 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 62/123] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:10.779378 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8d097-a3c7-44ce-b037-2a807b1f8158 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:10.780468 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c79960-d972-467b-8fc0-25651cbce7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:10.781136 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c79960-d972-467b-8fc0-25651cbce7dd None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:10.781438 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c79960-d972-467b-8fc0-25651cbce7dd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:10.781857 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c79960-d972-467b-8fc0-25651cbce7dd None admin] PATH_INFO: `/v3/projects/1570ef44f3134332b1fc668345a37479/users/164faaac1ec04c50ba5358bf58d221cf/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:10.784251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:10.790060 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c79960-d972-467b-8fc0-25651cbce7dd None admin] RBAC: Authorizing `identity:create_grant(project_id=1570ef44f3134332b1fc668345a37479, user_id=164faaac1ec04c50ba5358bf58d221cf, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:10.792008 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c79960-d972-467b-8fc0-25651cbce7dd None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:10.877639 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8d097-a3c7-44ce-b037-2a807b1f8158 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:10.914382 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 62/124] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 10 16:54:10 2022] PUT /identity/v3/projects/1570ef44f3134332b1fc668345a37479/users/164faaac1ec04c50ba5358bf58d221cf/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:10.914382 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c79960-d972-467b-8fc0-25651cbce7dd None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:10.921597 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a709c9-2be3-4a1c-96b2-1aafaebec086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:10.923088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a709c9-2be3-4a1c-96b2-1aafaebec086 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:10.923478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a709c9-2be3-4a1c-96b2-1aafaebec086 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:10.924131 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a709c9-2be3-4a1c-96b2-1aafaebec086 None admin] PATH_INFO: `/v3/users/7e545de4333749148bc08c29a9785d7c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:10.945904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a709c9-2be3-4a1c-96b2-1aafaebec086 None admin] RBAC: Authorizing `identity:get_user(user_id=7e545de4333749148bc08c29a9785d7c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:10.948287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a709c9-2be3-4a1c-96b2-1aafaebec086 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:10.952800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 63/125] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:10 2022] GET /identity/v3/users/7e545de4333749148bc08c29a9785d7c => generated 289 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:10.952800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a709c9-2be3-4a1c-96b2-1aafaebec086 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:11.045199 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:11.052044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c79960-d972-467b-8fc0-25651cbce7dd None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:11.093446 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611565d0-e447-4eab-a5e2-ca4724d5d4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:11.093945 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a709c9-2be3-4a1c-96b2-1aafaebec086 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:11.094063 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611565d0-e447-4eab-a5e2-ca4724d5d4fc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:11.094371 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611565d0-e447-4eab-a5e2-ca4724d5d4fc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:11.094646 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611565d0-e447-4eab-a5e2-ca4724d5d4fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:11.097606 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611565d0-e447-4eab-a5e2-ca4724d5d4fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:11.099132 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611565d0-e447-4eab-a5e2-ca4724d5d4fc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:11.100889 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 63/126] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:54:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:11.101894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611565d0-e447-4eab-a5e2-ca4724d5d4fc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:11.131213 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:11.140886 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d9e66-5621-455c-be77-bb78c5546107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:11.141529 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d9e66-5621-455c-be77-bb78c5546107 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:11.142500 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d9e66-5621-455c-be77-bb78c5546107 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:11.142500 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d9e66-5621-455c-be77-bb78c5546107 None admin] PATH_INFO: `/v3/projects/1570ef44f3134332b1fc668345a37479` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:11.143527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d9e66-5621-455c-be77-bb78c5546107 None admin] RBAC: Authorizing `identity:get_project(project_id=1570ef44f3134332b1fc668345a37479)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:11.145058 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d9e66-5621-455c-be77-bb78c5546107 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:11.150389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 64/127] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:10 2022] GET /identity/v3/projects/1570ef44f3134332b1fc668345a37479 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:11.150389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d9e66-5621-455c-be77-bb78c5546107 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:11.234920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611565d0-e447-4eab-a5e2-ca4724d5d4fc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:11.259428 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d9e66-5621-455c-be77-bb78c5546107 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:11.275954 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069679fa-16ef-47f1-a0bd-7b656d9557a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:11.276639 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069679fa-16ef-47f1-a0bd-7b656d9557a3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:11.276917 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069679fa-16ef-47f1-a0bd-7b656d9557a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:11.277237 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069679fa-16ef-47f1-a0bd-7b656d9557a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:11.288048 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069679fa-16ef-47f1-a0bd-7b656d9557a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:11.292908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-069679fa-16ef-47f1-a0bd-7b656d9557a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:11.292908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:11.292908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:11.292908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:11.292908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:11.292908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:11.292908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:11.292908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:11.292908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:11.292908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:11.292908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:11.292908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:11.292908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:11.292908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:11.292908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:11.292908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:11.294677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:11.294677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:11.294677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:11.294677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:11.294677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:11.294677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:11.294677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:11.294677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:11.294677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:11.294677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:11.294677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:11.294677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:11.294677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:11.294677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:11.294677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:11.294677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:11.295505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:11.295505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:11.295505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:11.295505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:11.295505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:11.295505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:11.295505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:11.295505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:11.295505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:11.295505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:11.295505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:11.295505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:54:11.295505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 64/128] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:11.296143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069679fa-16ef-47f1-a0bd-7b656d9557a3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:11.307108 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a385df4-024a-4e03-bccd-c29f544b4b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:11.307981 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a385df4-024a-4e03-bccd-c29f544b4b36 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:11.308781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a385df4-024a-4e03-bccd-c29f544b4b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:11.309923 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a385df4-024a-4e03-bccd-c29f544b4b36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:11.335396 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:11.391446 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a385df4-024a-4e03-bccd-c29f544b4b36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:11.393042 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a385df4-024a-4e03-bccd-c29f544b4b36 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:11.417077 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069679fa-16ef-47f1-a0bd-7b656d9557a3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:11.438735 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:11.462387 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320aa6b-ddee-4cd9-afab-43f5d7fe0e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:11.463604 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320aa6b-ddee-4cd9-afab-43f5d7fe0e25 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:11.464053 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320aa6b-ddee-4cd9-afab-43f5d7fe0e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:11.464747 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320aa6b-ddee-4cd9-afab-43f5d7fe0e25 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:11.466524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320aa6b-ddee-4cd9-afab-43f5d7fe0e25 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:11.468176 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320aa6b-ddee-4cd9-afab-43f5d7fe0e25 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:11.502587 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 65/129] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:54:11.502587 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a385df4-024a-4e03-bccd-c29f544b4b36 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:11.533524 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:11.562963 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 65/130] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 10 16:54:11 2022] POST /identity/v3/projects => generated 302 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:11.564221 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320aa6b-ddee-4cd9-afab-43f5d7fe0e25 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:11.683918 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a385df4-024a-4e03-bccd-c29f544b4b36 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:11.770510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320aa6b-ddee-4cd9-afab-43f5d7fe0e25 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:11.771075 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73683f-f8c6-458c-825d-f05028fddd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:11.771755 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73683f-f8c6-458c-825d-f05028fddd7f None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:11.772034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73683f-f8c6-458c-825d-f05028fddd7f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:11.772309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73683f-f8c6-458c-825d-f05028fddd7f None admin] PATH_INFO: `/v3/projects/1570ef44f3134332b1fc668345a37479/users/b98eacd3b6474daabcffaf3ef31aea2b/roles/a3e5816caacc425c9a6dd381fe0eddf0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:11.776927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73683f-f8c6-458c-825d-f05028fddd7f None admin] RBAC: Authorizing `identity:create_grant(project_id=1570ef44f3134332b1fc668345a37479, user_id=b98eacd3b6474daabcffaf3ef31aea2b, role_id=a3e5816caacc425c9a6dd381fe0eddf0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:11.778040 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73683f-f8c6-458c-825d-f05028fddd7f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:11.817715 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d4c6f-b111-4474-a6ef-6b29d3745906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:11.818227 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d4c6f-b111-4474-a6ef-6b29d3745906 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:11.818580 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d4c6f-b111-4474-a6ef-6b29d3745906 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:11.818855 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d4c6f-b111-4474-a6ef-6b29d3745906 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:11.819525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0d4c6f-b111-4474-a6ef-6b29d3745906 None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:11.819883 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0d4c6f-b111-4474-a6ef-6b29d3745906 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:11.821266 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0d4c6f-b111-4474-a6ef-6b29d3745906 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:11.822994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:11.832007 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 66/131] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:11.832675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d4c6f-b111-4474-a6ef-6b29d3745906 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:11.868021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 66/132] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 10 16:54:11 2022] PUT /identity/v3/projects/1570ef44f3134332b1fc668345a37479/users/b98eacd3b6474daabcffaf3ef31aea2b/roles/a3e5816caacc425c9a6dd381fe0eddf0 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:11.875357 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73683f-f8c6-458c-825d-f05028fddd7f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:11.982945 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d4c6f-b111-4474-a6ef-6b29d3745906 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:12.003398 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:12.014739 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73683f-f8c6-458c-825d-f05028fddd7f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:12.030584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2253ec4-dd8f-4daf-a644-91e23a96efd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:12.031492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2253ec4-dd8f-4daf-a644-91e23a96efd6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:12.031840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2253ec4-dd8f-4daf-a644-91e23a96efd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:12.032312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2253ec4-dd8f-4daf-a644-91e23a96efd6 None admin] PATH_INFO: `/v3/projects/1570ef44f3134332b1fc668345a37479` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:12.033745 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2253ec4-dd8f-4daf-a644-91e23a96efd6 None admin] RBAC: Authorizing `identity:get_project(project_id=1570ef44f3134332b1fc668345a37479)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:12.035765 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2253ec4-dd8f-4daf-a644-91e23a96efd6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:12.037946 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 67/133] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:11 2022] GET /identity/v3/projects/1570ef44f3134332b1fc668345a37479 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:12.039896 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb4983-5005-491c-b69b-99f1f8866441 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:12.040361 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb4983-5005-491c-b69b-99f1f8866441 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:12.040508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb4983-5005-491c-b69b-99f1f8866441 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:12.042225 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 68/134] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:12.043306 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb4983-5005-491c-b69b-99f1f8866441 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:12.061852 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd620a5d-d0e5-477f-8b87-dd9da3b93b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:12.062917 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd620a5d-d0e5-477f-8b87-dd9da3b93b35 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:12.063341 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd620a5d-d0e5-477f-8b87-dd9da3b93b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:12.063795 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd620a5d-d0e5-477f-8b87-dd9da3b93b35 None admin] PATH_INFO: `/v3/users/b98eacd3b6474daabcffaf3ef31aea2b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:12.065186 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd620a5d-d0e5-477f-8b87-dd9da3b93b35 None admin] RBAC: Authorizing `identity:get_user(user_id=b98eacd3b6474daabcffaf3ef31aea2b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:12.066982 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd620a5d-d0e5-477f-8b87-dd9da3b93b35 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:12.069956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 67/135] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:11 2022] GET /identity/v3/users/b98eacd3b6474daabcffaf3ef31aea2b => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:12.071619 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a2204-66a5-4b23-8a68-8c089b1e5f8d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:12.072047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a2204-66a5-4b23-8a68-8c089b1e5f8d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:12.072459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a2204-66a5-4b23-8a68-8c089b1e5f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:12.096101 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-984a2204-66a5-4b23-8a68-8c089b1e5f8d None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:12.097579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-984a2204-66a5-4b23-8a68-8c089b1e5f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:12.124293 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:12.177131 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb4983-5005-491c-b69b-99f1f8866441 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:12.181774 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 68/136] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:12.182853 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a2204-66a5-4b23-8a68-8c089b1e5f8d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:12.217735 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84603012-3d2a-4df5-bf74-04471a17536f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:12.219024 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84603012-3d2a-4df5-bf74-04471a17536f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:12.219414 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84603012-3d2a-4df5-bf74-04471a17536f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:12.219788 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84603012-3d2a-4df5-bf74-04471a17536f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:12.220806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84603012-3d2a-4df5-bf74-04471a17536f None admin] RBAC: Adding query filter params (role.id=dc46bbae6e0f42baaa61e88b45035c46, scope.project.id=1570ef44f3134332b1fc668345a37479, user.id=7e545de4333749148bc08c29a9785d7c) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:12.221275 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84603012-3d2a-4df5-bf74-04471a17536f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:12.223006 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84603012-3d2a-4df5-bf74-04471a17536f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:12.229834 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 69/137] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:54:12 2022] GET /identity/v3/role_assignments?user.id=7e545de4333749148bc08c29a9785d7c&scope.project.id=1570ef44f3134332b1fc668345a37479&role.id=dc46bbae6e0f42baaa61e88b45035c46 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:12.236194 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84603012-3d2a-4df5-bf74-04471a17536f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:12.327989 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:12.351324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a2204-66a5-4b23-8a68-8c089b1e5f8d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:12.361960 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84603012-3d2a-4df5-bf74-04471a17536f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:12.395446 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:12.404887 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b0bc56-42e3-47d9-81a5-e42123157050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:12.405768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0bc56-42e3-47d9-81a5-e42123157050 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:12.406056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0bc56-42e3-47d9-81a5-e42123157050 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:12.406333 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0bc56-42e3-47d9-81a5-e42123157050 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:12.420712 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b0bc56-42e3-47d9-81a5-e42123157050 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:12.425609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-18b0bc56-42e3-47d9-81a5-e42123157050 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:12.425609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:12.425609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:12.425609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:12.425609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:12.425609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:12.425609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:12.425609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:12.425609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:12.425609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:12.425609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:12.425609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:12.425609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:12.425609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:12.425609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:12.425609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:12.426506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:12.426506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:12.426506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:12.426506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:12.426506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:12.426506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:12.426506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:12.426506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:12.426506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:12.426506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:12.426506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:12.426506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:12.426506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:12.426506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:12.426506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:12.426506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:12.427605 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:12.427605 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:12.427605 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:12.427605 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:12.427605 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:12.427605 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:12.427605 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:12.427605 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:12.427605 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:12.427605 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:12.427605 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:12.427605 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:54:12.427605 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 70/138] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:12.429222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e775ad-1dc1-4954-ae0b-d43dc2bf6384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:12.430064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e775ad-1dc1-4954-ae0b-d43dc2bf6384 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:12.430530 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e775ad-1dc1-4954-ae0b-d43dc2bf6384 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:12.430985 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e775ad-1dc1-4954-ae0b-d43dc2bf6384 None admin] PATH_INFO: `/v3/projects/29443ed08048475986823123d96ec005` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:12.436032 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e775ad-1dc1-4954-ae0b-d43dc2bf6384 None admin] RBAC: Authorizing `identity:get_project(project_id=29443ed08048475986823123d96ec005)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:12.436197 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b0bc56-42e3-47d9-81a5-e42123157050 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:12.437939 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e775ad-1dc1-4954-ae0b-d43dc2bf6384 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:12.440450 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 69/139] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:12 2022] GET /identity/v3/projects/29443ed08048475986823123d96ec005 => generated 312 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:12.448581 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e775ad-1dc1-4954-ae0b-d43dc2bf6384 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:12.561324 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b0bc56-42e3-47d9-81a5-e42123157050 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:12.604575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb579cc1-32b7-4274-b132-ca7539f4f88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:12.605238 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb579cc1-32b7-4274-b132-ca7539f4f88a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:12.605520 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb579cc1-32b7-4274-b132-ca7539f4f88a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:12.605791 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb579cc1-32b7-4274-b132-ca7539f4f88a None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:12.608917 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb579cc1-32b7-4274-b132-ca7539f4f88a None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:12.609398 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb579cc1-32b7-4274-b132-ca7539f4f88a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:12.611776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb579cc1-32b7-4274-b132-ca7539f4f88a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:12.624261 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 71/140] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:12.630134 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb579cc1-32b7-4274-b132-ca7539f4f88a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:12.644637 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e775ad-1dc1-4954-ae0b-d43dc2bf6384 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:12.687686 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3617386-0b69-4d2f-b3fe-2c208cc33116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:12.688529 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3617386-0b69-4d2f-b3fe-2c208cc33116 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:12.688907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3617386-0b69-4d2f-b3fe-2c208cc33116 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:12.689302 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3617386-0b69-4d2f-b3fe-2c208cc33116 None admin] PATH_INFO: `/v3/projects/29443ed08048475986823123d96ec005/users/b98eacd3b6474daabcffaf3ef31aea2b/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:12.690700 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:12.693757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3617386-0b69-4d2f-b3fe-2c208cc33116 None admin] RBAC: Authorizing `identity:create_grant(project_id=29443ed08048475986823123d96ec005, user_id=b98eacd3b6474daabcffaf3ef31aea2b, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:12.695275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3617386-0b69-4d2f-b3fe-2c208cc33116 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:12.740848 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb579cc1-32b7-4274-b132-ca7539f4f88a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:12.775674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 70/141] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 10 16:54:12 2022] PUT /identity/v3/projects/29443ed08048475986823123d96ec005/users/b98eacd3b6474daabcffaf3ef31aea2b/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:12.778396 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc4748-a39d-4f4b-9c4b-e9ec6fbad7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:12.779242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc4748-a39d-4f4b-9c4b-e9ec6fbad7e1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:12.779521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc4748-a39d-4f4b-9c4b-e9ec6fbad7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:12.779793 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc4748-a39d-4f4b-9c4b-e9ec6fbad7e1 None admin] PATH_INFO: `/v3/users/b98eacd3b6474daabcffaf3ef31aea2b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:12.781053 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc4748-a39d-4f4b-9c4b-e9ec6fbad7e1 None admin] RBAC: Authorizing `identity:get_user(user_id=b98eacd3b6474daabcffaf3ef31aea2b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:12.782644 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc4748-a39d-4f4b-9c4b-e9ec6fbad7e1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:12.784551 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 72/142] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:12 2022] GET /identity/v3/users/b98eacd3b6474daabcffaf3ef31aea2b => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:12.790325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3617386-0b69-4d2f-b3fe-2c208cc33116 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:12.820020 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:12.936630 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3617386-0b69-4d2f-b3fe-2c208cc33116 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:12.987540 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe3ec9-c047-452c-b0e9-4e2535121b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:12.988271 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe3ec9-c047-452c-b0e9-4e2535121b93 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:12.988552 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe3ec9-c047-452c-b0e9-4e2535121b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:12.988827 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe3ec9-c047-452c-b0e9-4e2535121b93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:13.083273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe3ec9-c047-452c-b0e9-4e2535121b93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:13.084777 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe3ec9-c047-452c-b0e9-4e2535121b93 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:13.103487 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:13.193005 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 71/143] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:54:13.198658 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc4748-a39d-4f4b-9c4b-e9ec6fbad7e1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:13.319030 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea38e3-8777-41f7-9306-f888fc98e025 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:13.319242 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea38e3-8777-41f7-9306-f888fc98e025 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:13.319484 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea38e3-8777-41f7-9306-f888fc98e025 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:13.322982 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 72/144] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:13.331645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553d1b6-8ac7-4df6-9ddb-5437680378c8 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:13.331981 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553d1b6-8ac7-4df6-9ddb-5437680378c8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:13.332175 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553d1b6-8ac7-4df6-9ddb-5437680378c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:13.343291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc4748-a39d-4f4b-9c4b-e9ec6fbad7e1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:13.372595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:13.380117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6680f05-c24c-422f-8adc-09d58cee7107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:13.381000 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6680f05-c24c-422f-8adc-09d58cee7107 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:13.381499 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6680f05-c24c-422f-8adc-09d58cee7107 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:13.382008 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6680f05-c24c-422f-8adc-09d58cee7107 None admin] PATH_INFO: `/v3/projects/1570ef44f3134332b1fc668345a37479` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:13.383380 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4553d1b6-8ac7-4df6-9ddb-5437680378c8 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:13.383995 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6680f05-c24c-422f-8adc-09d58cee7107 None admin] RBAC: Authorizing `identity:get_project(project_id=1570ef44f3134332b1fc668345a37479)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:13.384594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4553d1b6-8ac7-4df6-9ddb-5437680378c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:13.385875 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6680f05-c24c-422f-8adc-09d58cee7107 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:13.388704 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 73/145] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:13 2022] GET /identity/v3/projects/1570ef44f3134332b1fc668345a37479 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:13.394460 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6680f05-c24c-422f-8adc-09d58cee7107 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:13.508768 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 73/146] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:13.549681 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553d1b6-8ac7-4df6-9ddb-5437680378c8 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:13.586201 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6680f05-c24c-422f-8adc-09d58cee7107 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:13.659104 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1cf31-6342-4854-a736-7ec5f77e90d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:13.659723 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1cf31-6342-4854-a736-7ec5f77e90d6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:13.660003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1cf31-6342-4854-a736-7ec5f77e90d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:13.660280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1cf31-6342-4854-a736-7ec5f77e90d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:13.661075 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a1cf31-6342-4854-a736-7ec5f77e90d6 None admin] RBAC: Adding query filter params (role.id=dfb72dc1f3744c8e957abcc4ac967dac, scope.project.id=1570ef44f3134332b1fc668345a37479, user.id=b98eacd3b6474daabcffaf3ef31aea2b) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:13.661388 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a1cf31-6342-4854-a736-7ec5f77e90d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:13.662868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a1cf31-6342-4854-a736-7ec5f77e90d6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:13.667450 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:13.674396 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 74/147] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:54:13 2022] GET /identity/v3/role_assignments?user.id=b98eacd3b6474daabcffaf3ef31aea2b&scope.project.id=1570ef44f3134332b1fc668345a37479&role.id=dfb72dc1f3744c8e957abcc4ac967dac => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:13.769209 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553d1b6-8ac7-4df6-9ddb-5437680378c8 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:13.817663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11bc1fc-53e9-4933-829c-e1f4c4667f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:13.818295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11bc1fc-53e9-4933-829c-e1f4c4667f11 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:13.818600 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11bc1fc-53e9-4933-829c-e1f4c4667f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:13.818882 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11bc1fc-53e9-4933-829c-e1f4c4667f11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:13.837795 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11bc1fc-53e9-4933-829c-e1f4c4667f11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:13.843351 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-c11bc1fc-53e9-4933-829c-e1f4c4667f11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:54:13.843351 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:13.843351 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:13.843351 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:13.843351 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:13.843351 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:13.843351 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:13.843351 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:13.843351 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:13.843351 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:13.843351 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:13.843351 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:13.843351 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:13.843351 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:13.843351 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:13.843351 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:13.844276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:13.844276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:13.844276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:13.844276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:13.844276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:13.844276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:13.844276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:13.844276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:13.844276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:13.844276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:13.844276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:13.844276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:13.844276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:13.844276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:13.844276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:13.844276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:13.845088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:13.845088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:13.845088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:13.845088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:13.845088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:13.845088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:13.845088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:13.845088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:13.845088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:13.845088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:13.845088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:54:13.845088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:13.845088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 74/148] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:54:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:13.851477 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1cf31-6342-4854-a736-7ec5f77e90d6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:13.942699 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:13.981984 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1cf31-6342-4854-a736-7ec5f77e90d6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:14.020475 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd842e-ab8b-4d63-b7b2-d156f393fae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:14.021009 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd842e-ab8b-4d63-b7b2-d156f393fae9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:14.021287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd842e-ab8b-4d63-b7b2-d156f393fae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:14.021559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd842e-ab8b-4d63-b7b2-d156f393fae9 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:14.022230 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd842e-ab8b-4d63-b7b2-d156f393fae9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:14.022557 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd842e-ab8b-4d63-b7b2-d156f393fae9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:14.023961 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd842e-ab8b-4d63-b7b2-d156f393fae9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:14.033844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 75/149] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:14.043553 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11bc1fc-53e9-4933-829c-e1f4c4667f11 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:14.188695 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11bc1fc-53e9-4933-829c-e1f4c4667f11 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:14.201198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:14.236882 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52141f4-32b4-4f63-8a6f-f3fe90716ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:14.237542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52141f4-32b4-4f63-8a6f-f3fe90716ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:14.237817 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52141f4-32b4-4f63-8a6f-f3fe90716ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:14.238086 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52141f4-32b4-4f63-8a6f-f3fe90716ba5 None admin] PATH_INFO: `/v3/users/164faaac1ec04c50ba5358bf58d221cf` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:14.240804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52141f4-32b4-4f63-8a6f-f3fe90716ba5 None admin] RBAC: Authorizing `identity:get_user(user_id=164faaac1ec04c50ba5358bf58d221cf)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:14.242312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52141f4-32b4-4f63-8a6f-f3fe90716ba5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:14.244494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 75/150] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:14 2022] GET /identity/v3/users/164faaac1ec04c50ba5358bf58d221cf => generated 247 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:14.250378 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd842e-ab8b-4d63-b7b2-d156f393fae9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:14.422294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd842e-ab8b-4d63-b7b2-d156f393fae9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:14.440962 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:14.466344 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05459a4-c3f9-49f5-ae2d-f8d61e1a2ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:14.466968 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05459a4-c3f9-49f5-ae2d-f8d61e1a2ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:14.467248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05459a4-c3f9-49f5-ae2d-f8d61e1a2ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:14.467518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05459a4-c3f9-49f5-ae2d-f8d61e1a2ae3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:14.557809 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05459a4-c3f9-49f5-ae2d-f8d61e1a2ae3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:14.559339 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05459a4-c3f9-49f5-ae2d-f8d61e1a2ae3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:14.580444 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef599f-47d4-4d12-8395-c87f26029d2f None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:14.580977 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef599f-47d4-4d12-8395-c87f26029d2f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:14.580977 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef599f-47d4-4d12-8395-c87f26029d2f None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:14.582314 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 76/151] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:14.587322 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887725de-3c80-4108-9811-9d4b88dad60f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:14.587601 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887725de-3c80-4108-9811-9d4b88dad60f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:14.587872 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887725de-3c80-4108-9811-9d4b88dad60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:14.614649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-887725de-3c80-4108-9811-9d4b88dad60f None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:14.614649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-887725de-3c80-4108-9811-9d4b88dad60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:14.712928 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 77/152] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:54:14.738403 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887725de-3c80-4108-9811-9d4b88dad60f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:14.785148 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 76/153] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:54:14.801498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05459a4-c3f9-49f5-ae2d-f8d61e1a2ae3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:14.833064 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:14.849645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887725de-3c80-4108-9811-9d4b88dad60f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:14.873743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:14.885488 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72177a7-4799-4db7-9108-d69f5cce60e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:14.886252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72177a7-4799-4db7-9108-d69f5cce60e4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:14.886660 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72177a7-4799-4db7-9108-d69f5cce60e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:14.887028 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72177a7-4799-4db7-9108-d69f5cce60e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:14.888310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72177a7-4799-4db7-9108-d69f5cce60e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:14.889891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72177a7-4799-4db7-9108-d69f5cce60e4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:14.892150 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 78/154] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:54:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:14.897588 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72177a7-4799-4db7-9108-d69f5cce60e4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:15.015180 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05459a4-c3f9-49f5-ae2d-f8d61e1a2ae3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:15.084918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db0ec2-b8b2-4035-ba61-c452a0d48807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:15.085824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db0ec2-b8b2-4035-ba61-c452a0d48807 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:15.086183 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72177a7-4799-4db7-9108-d69f5cce60e4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:15.086429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db0ec2-b8b2-4035-ba61-c452a0d48807 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:15.086840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db0ec2-b8b2-4035-ba61-c452a0d48807 None admin] PATH_INFO: `/v3/projects/1570ef44f3134332b1fc668345a37479` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:15.089368 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6db0ec2-b8b2-4035-ba61-c452a0d48807 None admin] RBAC: Authorizing `identity:get_project(project_id=1570ef44f3134332b1fc668345a37479)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:15.091758 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6db0ec2-b8b2-4035-ba61-c452a0d48807 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:15.093601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 77/155] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:14 2022] GET /identity/v3/projects/1570ef44f3134332b1fc668345a37479 => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:15.095499 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0778b8e-611e-4d65-8504-43cdd55590be None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:15.095928 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0778b8e-611e-4d65-8504-43cdd55590be None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:15.096804 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0778b8e-611e-4d65-8504-43cdd55590be None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:15.098659 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 78/156] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:15.103478 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0778b8e-611e-4d65-8504-43cdd55590be None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:15.129780 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdd3b0-3102-405d-a752-a855b839d519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:15.130490 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdd3b0-3102-405d-a752-a855b839d519 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:15.130771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdd3b0-3102-405d-a752-a855b839d519 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:15.131043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdd3b0-3102-405d-a752-a855b839d519 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:15.131761 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bdd3b0-3102-405d-a752-a855b839d519 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:15.133996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bdd3b0-3102-405d-a752-a855b839d519 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:15.198161 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:15.219774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0778b8e-611e-4d65-8504-43cdd55590be None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:15.220239 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 79/157] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 10 16:54:14 2022] POST /identity/v3/users => generated 283 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:15.223071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c8a18-e64b-4ecc-88c9-fa9a363e0284 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:15.223611 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c8a18-e64b-4ecc-88c9-fa9a363e0284 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:15.224091 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c8a18-e64b-4ecc-88c9-fa9a363e0284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:15.250486 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-032c8a18-e64b-4ecc-88c9-fa9a363e0284 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:15.252994 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-032c8a18-e64b-4ecc-88c9-fa9a363e0284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:15.268738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966605e-1934-413d-a9ed-9f43f1505775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:15.269271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966605e-1934-413d-a9ed-9f43f1505775 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:15.269544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966605e-1934-413d-a9ed-9f43f1505775 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:15.269806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966605e-1934-413d-a9ed-9f43f1505775 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:15.270607 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2966605e-1934-413d-a9ed-9f43f1505775 None admin] RBAC: Adding query filter params (role.id=7554b353873942fab506de830188aba0, scope.project.id=1570ef44f3134332b1fc668345a37479, user.id=164faaac1ec04c50ba5358bf58d221cf) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:15.270967 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2966605e-1934-413d-a9ed-9f43f1505775 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:15.272694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2966605e-1934-413d-a9ed-9f43f1505775 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:15.280948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 79/158] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:54:15 2022] GET /identity/v3/role_assignments?user.id=164faaac1ec04c50ba5358bf58d221cf&scope.project.id=1570ef44f3134332b1fc668345a37479&role.id=7554b353873942fab506de830188aba0 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:15.282013 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fc4df-bff3-4b9b-8f07-75d318a15189 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:15.282294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fc4df-bff3-4b9b-8f07-75d318a15189 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:15.282954 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fc4df-bff3-4b9b-8f07-75d318a15189 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:15.284013 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 80/159] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:15.290183 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd225dc-0cda-437b-93f0-2e607911690a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:15.290511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd225dc-0cda-437b-93f0-2e607911690a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:15.290769 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd225dc-0cda-437b-93f0-2e607911690a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:15.291371 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:15.312548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-cbd225dc-0cda-437b-93f0-2e607911690a None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:15.313769 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cbd225dc-0cda-437b-93f0-2e607911690a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:15.355627 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 80/160] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:15.408439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c8a18-e64b-4ecc-88c9-fa9a363e0284 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:15.454079 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 81/161] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:54:15.477206 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd225dc-0cda-437b-93f0-2e607911690a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:15.557760 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c8a18-e64b-4ecc-88c9-fa9a363e0284 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:15.584254 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:15.614304 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee35addf-b1d8-49af-b2b8-9e2f5e68905d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:15.614773 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35addf-b1d8-49af-b2b8-9e2f5e68905d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:15.614854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35addf-b1d8-49af-b2b8-9e2f5e68905d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:15.615486 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35addf-b1d8-49af-b2b8-9e2f5e68905d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:15.633780 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee35addf-b1d8-49af-b2b8-9e2f5e68905d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:15.634495 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd225dc-0cda-437b-93f0-2e607911690a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:15.641542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-ee35addf-b1d8-49af-b2b8-9e2f5e68905d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:54:15.641542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:15.641542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:15.641542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:15.641542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:15.641542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:15.641542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:15.641542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:15.641542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:15.641542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:15.641542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:15.641542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:15.641542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:15.641542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:15.641542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:15.641542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:15.642571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:15.642571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:15.642571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:15.642571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:15.642571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:15.642571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:15.642571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:15.642571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:15.642571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:15.642571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:15.642571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:15.642571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:15.642571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:15.642571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:15.642571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:15.642571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:15.643402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:15.643402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:15.643402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:15.643402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:15.643402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:15.643402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:15.643402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:15.643402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:15.643402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:15.643402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:15.643402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:54:15.643402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:15.645245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 81/162] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 10 16:54:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:15.653919 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee35addf-b1d8-49af-b2b8-9e2f5e68905d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:15.686214 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:15.690039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d85022e-ee29-4309-85ad-0daa3c5c0089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:15.691784 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85022e-ee29-4309-85ad-0daa3c5c0089 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:15.692130 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85022e-ee29-4309-85ad-0daa3c5c0089 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:15.692464 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85022e-ee29-4309-85ad-0daa3c5c0089 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:15.712316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d85022e-ee29-4309-85ad-0daa3c5c0089 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:15.717455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2d85022e-ee29-4309-85ad-0daa3c5c0089 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:54:15.717455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:15.717455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:15.717455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:15.717455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:15.717455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:15.717455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:15.717455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:15.717455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:15.717455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:15.717455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:15.717455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:15.717455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:15.717455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:15.717455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:15.717455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:15.718548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:15.718548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:15.718548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:15.718548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:15.718548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:15.718548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:15.718548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:15.718548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:15.718548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:15.718548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:15.718548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:15.718548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:15.718548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:15.718548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:15.718548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:15.718548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:15.719316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:15.719316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:15.719316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:15.719316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:15.719316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:15.719316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:15.719316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:15.719316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:15.719316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:15.719316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:15.719316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:54:15.719316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:54:15.723849 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 82/163] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:54:15.730811 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d85022e-ee29-4309-85ad-0daa3c5c0089 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:15.794799 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee35addf-b1d8-49af-b2b8-9e2f5e68905d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:15.834250 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71fc50-d5af-46bb-aa62-6058ce852225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:15.835044 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71fc50-d5af-46bb-aa62-6058ce852225 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:15.835419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71fc50-d5af-46bb-aa62-6058ce852225 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:15.835804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71fc50-d5af-46bb-aa62-6058ce852225 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:15.836576 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71fc50-d5af-46bb-aa62-6058ce852225 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:15.837007 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71fc50-d5af-46bb-aa62-6058ce852225 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:15.839294 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71fc50-d5af-46bb-aa62-6058ce852225 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:15.848470 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 82/164] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:54:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:15.849359 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d85022e-ee29-4309-85ad-0daa3c5c0089 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:15.853984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71fc50-d5af-46bb-aa62-6058ce852225 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:15.892195 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951bf274-f136-4087-b90a-49d385f94daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:15.893361 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bf274-f136-4087-b90a-49d385f94daf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:15.893828 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bf274-f136-4087-b90a-49d385f94daf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:15.894094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bf274-f136-4087-b90a-49d385f94daf None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:15.894814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951bf274-f136-4087-b90a-49d385f94daf None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:15.895108 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951bf274-f136-4087-b90a-49d385f94daf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:15.896511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951bf274-f136-4087-b90a-49d385f94daf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:15.898444 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:15.907902 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 83/165] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:15.909128 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5255f97-878b-47ba-9404-26c15cc94c98 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:15.910563 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5255f97-878b-47ba-9404-26c15cc94c98 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:15.910563 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5255f97-878b-47ba-9404-26c15cc94c98 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:15.911301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 84/166] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:15.915848 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5255f97-878b-47ba-9404-26c15cc94c98 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:15.957275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:15.983824 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71fc50-d5af-46bb-aa62-6058ce852225 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:16.030509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d6f24-71f3-459c-a10b-2c54539c1869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:16.031504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d6f24-71f3-459c-a10b-2c54539c1869 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:16.032061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d6f24-71f3-459c-a10b-2c54539c1869 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:16.032533 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d6f24-71f3-459c-a10b-2c54539c1869 None admin] PATH_INFO: `/v3/users/b98eacd3b6474daabcffaf3ef31aea2b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:16.034035 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6d6f24-71f3-459c-a10b-2c54539c1869 None admin] RBAC: Authorizing `identity:get_user(user_id=b98eacd3b6474daabcffaf3ef31aea2b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:16.035900 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6d6f24-71f3-459c-a10b-2c54539c1869 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:16.037839 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 83/167] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:15 2022] GET /identity/v3/users/b98eacd3b6474daabcffaf3ef31aea2b => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:16.039342 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87416a82-abed-4130-bd84-aefd1e5e3816 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:16.039840 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87416a82-abed-4130-bd84-aefd1e5e3816 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:16.040370 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87416a82-abed-4130-bd84-aefd1e5e3816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:16.067600 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-87416a82-abed-4130-bd84-aefd1e5e3816 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:16.068208 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5255f97-878b-47ba-9404-26c15cc94c98 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:16.068854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-87416a82-abed-4130-bd84-aefd1e5e3816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:16.111060 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e16942e-1f87-4b7c-8210-4bcdd5cfbdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:16.111927 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16942e-1f87-4b7c-8210-4bcdd5cfbdc6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:16.112361 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16942e-1f87-4b7c-8210-4bcdd5cfbdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:16.112874 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16942e-1f87-4b7c-8210-4bcdd5cfbdc6 None admin] PATH_INFO: `/v3/users/7e545de4333749148bc08c29a9785d7c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:16.114273 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e16942e-1f87-4b7c-8210-4bcdd5cfbdc6 None admin] RBAC: Authorizing `identity:get_user(user_id=7e545de4333749148bc08c29a9785d7c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:16.116012 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e16942e-1f87-4b7c-8210-4bcdd5cfbdc6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:16.118271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 85/168] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:15 2022] GET /identity/v3/users/7e545de4333749148bc08c29a9785d7c => generated 289 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:16.119332 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e16942e-1f87-4b7c-8210-4bcdd5cfbdc6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:16.163002 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 84/169] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:16.164101 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87416a82-abed-4130-bd84-aefd1e5e3816 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:16.215722 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:16.257021 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e16942e-1f87-4b7c-8210-4bcdd5cfbdc6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:16.257278 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:16.280861 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87416a82-abed-4130-bd84-aefd1e5e3816 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:16.293655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9247115-43f9-4512-8544-3e4d273df240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:16.294250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9247115-43f9-4512-8544-3e4d273df240 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:16.294575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9247115-43f9-4512-8544-3e4d273df240 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:16.294864 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9247115-43f9-4512-8544-3e4d273df240 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:16.319271 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df5c48-833d-42b9-b6d3-1a91ed91bb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:16.320195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df5c48-833d-42b9-b6d3-1a91ed91bb61 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:16.321035 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df5c48-833d-42b9-b6d3-1a91ed91bb61 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:16.321458 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df5c48-833d-42b9-b6d3-1a91ed91bb61 None admin] PATH_INFO: `/v3/projects/1570ef44f3134332b1fc668345a37479` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:16.322774 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df5c48-833d-42b9-b6d3-1a91ed91bb61 None admin] RBAC: Authorizing `identity:get_project(project_id=1570ef44f3134332b1fc668345a37479)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:16.324350 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df5c48-833d-42b9-b6d3-1a91ed91bb61 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:16.327024 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 85/170] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:16 2022] GET /identity/v3/projects/1570ef44f3134332b1fc668345a37479 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:16.327929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df5c48-833d-42b9-b6d3-1a91ed91bb61 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:16.375775 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9247115-43f9-4512-8544-3e4d273df240 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:16.377166 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9247115-43f9-4512-8544-3e4d273df240 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:16.441147 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df5c48-833d-42b9-b6d3-1a91ed91bb61 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:16.450717 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:16.499508 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac4fc5-a001-485c-8602-f2e584e8b9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:16.500795 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac4fc5-a001-485c-8602-f2e584e8b9ee None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:16.501749 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac4fc5-a001-485c-8602-f2e584e8b9ee None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:16.502621 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac4fc5-a001-485c-8602-f2e584e8b9ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:16.517540 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 86/171] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:54:16.519627 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9247115-43f9-4512-8544-3e4d273df240 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:16.525691 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ac4fc5-a001-485c-8602-f2e584e8b9ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:16.536447 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-82ac4fc5-a001-485c-8602-f2e584e8b9ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:16.536447 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:16.536447 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:16.536447 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:16.536447 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:16.536447 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:16.536447 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:16.536447 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:16.536447 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:16.536447 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:16.536447 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:16.536447 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:16.536447 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:16.536447 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:16.536447 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:16.536447 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:16.537582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:16.537582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:16.537582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:16.537582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:16.537582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:16.537582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:16.537582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:16.537582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:16.537582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:16.537582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:16.537582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:16.537582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:16.537582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:16.537582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:16.537582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:16.537582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:16.538579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:16.538579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:16.538579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:16.538579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:16.538579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:16.538579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:16.538579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:16.538579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:16.538579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:16.538579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:16.538579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:16.538579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:16.542387 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 86/172] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:16.543882 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac4fc5-a001-485c-8602-f2e584e8b9ee None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:16.560487 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:16.717344 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9247115-43f9-4512-8544-3e4d273df240 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:16.740403 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac4fc5-a001-485c-8602-f2e584e8b9ee None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:16.785781 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd792da5-dade-470b-b096-403450a9ee11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:16.786464 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd792da5-dade-470b-b096-403450a9ee11 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:16.786737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd792da5-dade-470b-b096-403450a9ee11 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:16.787000 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd792da5-dade-470b-b096-403450a9ee11 None admin] PATH_INFO: `/v3/projects/1570ef44f3134332b1fc668345a37479/users/7e545de4333749148bc08c29a9785d7c/roles/dc46bbae6e0f42baaa61e88b45035c46` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:16.790068 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd792da5-dade-470b-b096-403450a9ee11 None admin] RBAC: Authorizing `identity:create_grant(project_id=1570ef44f3134332b1fc668345a37479, user_id=7e545de4333749148bc08c29a9785d7c, role_id=dc46bbae6e0f42baaa61e88b45035c46)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:16.791545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd792da5-dade-470b-b096-403450a9ee11 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:16.802801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a423047-659e-4da1-845e-4b597479f203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:16.803422 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a423047-659e-4da1-845e-4b597479f203 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:16.803694 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a423047-659e-4da1-845e-4b597479f203 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:16.804016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a423047-659e-4da1-845e-4b597479f203 None admin] PATH_INFO: `/v3/projects/1570ef44f3134332b1fc668345a37479` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:16.805134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a423047-659e-4da1-845e-4b597479f203 None admin] RBAC: Authorizing `identity:get_project(project_id=1570ef44f3134332b1fc668345a37479)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:16.806740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a423047-659e-4da1-845e-4b597479f203 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:16.813214 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 87/173] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:16 2022] GET /identity/v3/projects/1570ef44f3134332b1fc668345a37479 => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:16.813214 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a423047-659e-4da1-845e-4b597479f203 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:16.868177 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 87/174] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 10 16:54:16 2022] PUT /identity/v3/projects/1570ef44f3134332b1fc668345a37479/users/7e545de4333749148bc08c29a9785d7c/roles/dc46bbae6e0f42baaa61e88b45035c46 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:16.869173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd792da5-dade-470b-b096-403450a9ee11 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:16.928499 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:16.930752 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:16.986625 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a423047-659e-4da1-845e-4b597479f203 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:17.043753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd792da5-dade-470b-b096-403450a9ee11 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:17.087165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4a491-3ccb-470c-82c0-6408ca2ef8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:17.087831 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4a491-3ccb-470c-82c0-6408ca2ef8bd None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:17.088104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4a491-3ccb-470c-82c0-6408ca2ef8bd None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:17.088368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4a491-3ccb-470c-82c0-6408ca2ef8bd None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:17.089056 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4a491-3ccb-470c-82c0-6408ca2ef8bd None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:17.089348 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4a491-3ccb-470c-82c0-6408ca2ef8bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:17.090823 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4a491-3ccb-470c-82c0-6408ca2ef8bd None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:17.090936 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a718cf35-c397-4de8-9fbf-3a7bcc468f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:17.091485 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718cf35-c397-4de8-9fbf-3a7bcc468f61 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:17.091769 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718cf35-c397-4de8-9fbf-3a7bcc468f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:17.092040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718cf35-c397-4de8-9fbf-3a7bcc468f61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:17.093264 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a718cf35-c397-4de8-9fbf-3a7bcc468f61 None admin] RBAC: Adding query filter params (role.id=a3e5816caacc425c9a6dd381fe0eddf0, scope.project.id=1570ef44f3134332b1fc668345a37479, user.id=b98eacd3b6474daabcffaf3ef31aea2b) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:17.093840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a718cf35-c397-4de8-9fbf-3a7bcc468f61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:17.095596 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a718cf35-c397-4de8-9fbf-3a7bcc468f61 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:17.103669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 88/175] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:54:16 2022] GET /identity/v3/role_assignments?user.id=b98eacd3b6474daabcffaf3ef31aea2b&scope.project.id=1570ef44f3134332b1fc668345a37479&role.id=a3e5816caacc425c9a6dd381fe0eddf0 => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:17.110019 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 88/176] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:17.116009 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a718cf35-c397-4de8-9fbf-3a7bcc468f61 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:17.265341 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:17.280823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a718cf35-c397-4de8-9fbf-3a7bcc468f61 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:17.324111 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb5f63-7aa5-4e22-b876-fed4523c123b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:17.325207 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb5f63-7aa5-4e22-b876-fed4523c123b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:17.325490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb5f63-7aa5-4e22-b876-fed4523c123b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:17.325766 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb5f63-7aa5-4e22-b876-fed4523c123b None admin] PATH_INFO: `/v3/users/b98eacd3b6474daabcffaf3ef31aea2b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:17.326948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fb5f63-7aa5-4e22-b876-fed4523c123b None admin] RBAC: Authorizing `identity:get_user(user_id=b98eacd3b6474daabcffaf3ef31aea2b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:17.328427 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fb5f63-7aa5-4e22-b876-fed4523c123b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:17.330258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 89/177] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:17 2022] GET /identity/v3/users/b98eacd3b6474daabcffaf3ef31aea2b => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:17.335955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4a491-3ccb-470c-82c0-6408ca2ef8bd None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:17.453343 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4a491-3ccb-470c-82c0-6408ca2ef8bd None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:17.489323 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ce8bb-6271-49c3-a831-04d6aea53bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:17.489931 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ce8bb-6271-49c3-a831-04d6aea53bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:17.490209 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ce8bb-6271-49c3-a831-04d6aea53bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:17.490512 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ce8bb-6271-49c3-a831-04d6aea53bd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:17.493987 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:17.563022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ce8bb-6271-49c3-a831-04d6aea53bd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:17.564408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ce8bb-6271-49c3-a831-04d6aea53bd6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:17.667950 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 89/178] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:54:17.673856 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb5f63-7aa5-4e22-b876-fed4523c123b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:17.814262 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb5f63-7aa5-4e22-b876-fed4523c123b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:17.855739 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed3ba7-6d89-48f1-88e3-a872e2cf4b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:17.856814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed3ba7-6d89-48f1-88e3-a872e2cf4b41 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:17.857305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed3ba7-6d89-48f1-88e3-a872e2cf4b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:17.857775 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed3ba7-6d89-48f1-88e3-a872e2cf4b41 None admin] PATH_INFO: `/v3/projects/29443ed08048475986823123d96ec005` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:17.863220 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed3ba7-6d89-48f1-88e3-a872e2cf4b41 None admin] RBAC: Authorizing `identity:get_project(project_id=29443ed08048475986823123d96ec005)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:17.864947 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed3ba7-6d89-48f1-88e3-a872e2cf4b41 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:17.867008 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 90/179] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:17 2022] GET /identity/v3/projects/29443ed08048475986823123d96ec005 => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:17.875482 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ce8bb-6271-49c3-a831-04d6aea53bd6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:17.938830 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:17.970547 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74eb71-12b2-4a2c-99df-e23ba9857540 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:17.971081 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74eb71-12b2-4a2c-99df-e23ba9857540 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:17.971081 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74eb71-12b2-4a2c-99df-e23ba9857540 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:17.972431 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 91/180] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:17.977537 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b11f74-ccfb-44cd-b8c4-daa9f41d8107 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:17.981334 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b11f74-ccfb-44cd-b8c4-daa9f41d8107 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:17.981334 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b11f74-ccfb-44cd-b8c4-daa9f41d8107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:18.012227 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ce8bb-6271-49c3-a831-04d6aea53bd6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:18.019588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-76b11f74-ccfb-44cd-b8c4-daa9f41d8107 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:18.021875 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-76b11f74-ccfb-44cd-b8c4-daa9f41d8107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:18.051802 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722054e6-4013-4a09-90d4-5f7e44f1e5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:18.052401 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722054e6-4013-4a09-90d4-5f7e44f1e5a3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:18.053640 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722054e6-4013-4a09-90d4-5f7e44f1e5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:18.053819 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722054e6-4013-4a09-90d4-5f7e44f1e5a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:18.055309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722054e6-4013-4a09-90d4-5f7e44f1e5a3 None admin] RBAC: Adding query filter params (role.id=dfb72dc1f3744c8e957abcc4ac967dac, scope.project.id=29443ed08048475986823123d96ec005, user.id=b98eacd3b6474daabcffaf3ef31aea2b) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:18.055619 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722054e6-4013-4a09-90d4-5f7e44f1e5a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:18.057049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722054e6-4013-4a09-90d4-5f7e44f1e5a3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:18.061260 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:18.072426 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 90/181] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:54:17 2022] GET /identity/v3/role_assignments?user.id=b98eacd3b6474daabcffaf3ef31aea2b&scope.project.id=29443ed08048475986823123d96ec005&role.id=dfb72dc1f3744c8e957abcc4ac967dac => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:18.134384 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 92/182] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:54:18.163255 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722054e6-4013-4a09-90d4-5f7e44f1e5a3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:18.284281 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722054e6-4013-4a09-90d4-5f7e44f1e5a3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:18.291820 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:18.321063 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726afdb-9c15-4086-a47d-f95885e036bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:18.321770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726afdb-9c15-4086-a47d-f95885e036bb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:18.322049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726afdb-9c15-4086-a47d-f95885e036bb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:18.322323 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726afdb-9c15-4086-a47d-f95885e036bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:18.334649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726afdb-9c15-4086-a47d-f95885e036bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:18.339456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-f726afdb-9c15-4086-a47d-f95885e036bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:54:18.339456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:18.339456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:18.339456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:18.339456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:18.339456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:18.339456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:18.339456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:18.339456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:18.339456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:18.339456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:18.339456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:18.339456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:18.339456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:18.339456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:18.339456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:18.340960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:18.340960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:18.340960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:18.340960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:18.340960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:18.340960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:18.340960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:18.340960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:18.340960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:18.340960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:18.340960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:18.340960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:18.340960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:18.340960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:18.340960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:18.340960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:18.341904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:18.341904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:18.341904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:18.341904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:18.341904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:18.341904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:18.341904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:18.341904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:18.341904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:18.341904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:18.341904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:54:18.341904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:18.341904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 91/183] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:54:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:18.347313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b11f74-ccfb-44cd-b8c4-daa9f41d8107 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:18.433038 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a01ea-f66b-4c17-a0ea-5b777851177e None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:18.433377 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a01ea-f66b-4c17-a0ea-5b777851177e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:18.433582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a01ea-f66b-4c17-a0ea-5b777851177e None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:18.440081 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 92/184] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:54:18.446908 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd7cb9-0363-4664-8521-71498e223478 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:18.447302 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd7cb9-0363-4664-8521-71498e223478 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:18.447737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd7cb9-0363-4664-8521-71498e223478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:18.462408 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b11f74-ccfb-44cd-b8c4-daa9f41d8107 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:18.470477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f4cd7cb9-0363-4664-8521-71498e223478 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:18.471981 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f4cd7cb9-0363-4664-8521-71498e223478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:18.504128 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0d496-35d2-4f23-99bc-ca92080f1066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:18.504981 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0d496-35d2-4f23-99bc-ca92080f1066 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:18.505396 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0d496-35d2-4f23-99bc-ca92080f1066 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:18.505794 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0d496-35d2-4f23-99bc-ca92080f1066 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:18.506800 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0d496-35d2-4f23-99bc-ca92080f1066 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:18.507246 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0d496-35d2-4f23-99bc-ca92080f1066 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:18.508775 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0d496-35d2-4f23-99bc-ca92080f1066 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:18.518706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 93/185] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:18.524165 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0d496-35d2-4f23-99bc-ca92080f1066 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:18.549526 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:18.556949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 93/186] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:18.582539 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd7cb9-0363-4664-8521-71498e223478 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:18.667373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0d496-35d2-4f23-99bc-ca92080f1066 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:18.707224 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:18.712203 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca922b-c3fb-4387-ab4c-b8b7572b88c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:18.713198 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca922b-c3fb-4387-ab4c-b8b7572b88c4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:18.713657 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca922b-c3fb-4387-ab4c-b8b7572b88c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:18.714129 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca922b-c3fb-4387-ab4c-b8b7572b88c4 None admin] PATH_INFO: `/v3/users/166e77bb8b8f412f92ffdd1c1d42661f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:18.720124 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd7cb9-0363-4664-8521-71498e223478 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:18.730507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca922b-c3fb-4387-ab4c-b8b7572b88c4 None admin] RBAC: Authorizing `identity:get_user(user_id=166e77bb8b8f412f92ffdd1c1d42661f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:18.733023 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca922b-c3fb-4387-ab4c-b8b7572b88c4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:18.734939 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 94/187] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:18 2022] GET /identity/v3/users/166e77bb8b8f412f92ffdd1c1d42661f => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:18.740569 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca922b-c3fb-4387-ab4c-b8b7572b88c4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:18.762611 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a7ccc9-05fc-4a0f-960d-9e4c1c8f5176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:18.763248 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7ccc9-05fc-4a0f-960d-9e4c1c8f5176 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:18.763525 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7ccc9-05fc-4a0f-960d-9e4c1c8f5176 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:18.763794 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7ccc9-05fc-4a0f-960d-9e4c1c8f5176 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:18.778749 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a7ccc9-05fc-4a0f-960d-9e4c1c8f5176 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:18.784567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-35a7ccc9-05fc-4a0f-960d-9e4c1c8f5176 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:54:18.784567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:18.784567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:18.784567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:18.784567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:18.784567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:18.784567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:18.784567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:18.784567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:18.784567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:18.784567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:18.784567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:18.784567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:18.784567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:18.784567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:18.784567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:18.785668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:18.785668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:18.785668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:18.785668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:18.785668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:18.785668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:18.785668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:18.785668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:18.785668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:18.785668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:18.785668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:18.785668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:18.785668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:18.785668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:18.785668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:18.785668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:18.786807 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:18.786807 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:18.786807 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:18.786807 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:18.786807 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:18.786807 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:18.786807 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:18.786807 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:18.786807 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:18.786807 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:18.786807 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:54:18.786807 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:18.786807 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 94/188] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 10 16:54:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:18.791632 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a7ccc9-05fc-4a0f-960d-9e4c1c8f5176 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:18.830654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:18.858243 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca922b-c3fb-4387-ab4c-b8b7572b88c4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:18.897034 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ab339-35de-4acd-b74b-0e3dc30ab78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:18.897846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a7ccc9-05fc-4a0f-960d-9e4c1c8f5176 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:18.898496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ab339-35de-4acd-b74b-0e3dc30ab78e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:18.898955 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ab339-35de-4acd-b74b-0e3dc30ab78e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:18.899489 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ab339-35de-4acd-b74b-0e3dc30ab78e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:18.935730 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa4dc0-6359-4b1f-940f-dfbe0d7eba83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:18.936342 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa4dc0-6359-4b1f-940f-dfbe0d7eba83 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:18.936621 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa4dc0-6359-4b1f-940f-dfbe0d7eba83 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:18.936889 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa4dc0-6359-4b1f-940f-dfbe0d7eba83 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:18.937558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa4dc0-6359-4b1f-940f-dfbe0d7eba83 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:18.937858 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa4dc0-6359-4b1f-940f-dfbe0d7eba83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:18.939276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa4dc0-6359-4b1f-940f-dfbe0d7eba83 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:18.947725 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 95/189] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:54:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:18.951832 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08324201-58bb-4d76-9dfe-5ffbea7d850b None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:18.952595 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08324201-58bb-4d76-9dfe-5ffbea7d850b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:18.952595 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08324201-58bb-4d76-9dfe-5ffbea7d850b None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:18.953847 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 96/190] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:18.954571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08324201-58bb-4d76-9dfe-5ffbea7d850b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:18.970251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:18.979617 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ab339-35de-4acd-b74b-0e3dc30ab78e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:18.981839 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ab339-35de-4acd-b74b-0e3dc30ab78e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:19.063721 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:19.068194 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08324201-58bb-4d76-9dfe-5ffbea7d850b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:19.089883 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 95/191] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 16:54:19.089883 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd932e77-24ef-48d5-88f1-7188523e23b8 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:19.089883 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd932e77-24ef-48d5-88f1-7188523e23b8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:19.089883 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd932e77-24ef-48d5-88f1-7188523e23b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:19.107762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2bc82-5e8d-4aaa-aa7d-46295a71a51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:19.108389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2bc82-5e8d-4aaa-aa7d-46295a71a51d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:19.108666 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2bc82-5e8d-4aaa-aa7d-46295a71a51d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:19.108927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2bc82-5e8d-4aaa-aa7d-46295a71a51d None admin] PATH_INFO: `/v3/users/166e77bb8b8f412f92ffdd1c1d42661f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:19.110087 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2bc82-5e8d-4aaa-aa7d-46295a71a51d None admin] RBAC: Authorizing `identity:get_user(user_id=166e77bb8b8f412f92ffdd1c1d42661f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:19.111559 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2bc82-5e8d-4aaa-aa7d-46295a71a51d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:19.112547 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-dd932e77-24ef-48d5-88f1-7188523e23b8 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:19.113365 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 97/192] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:18 2022] GET /identity/v3/users/166e77bb8b8f412f92ffdd1c1d42661f => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:19.113919 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2bc82-5e8d-4aaa-aa7d-46295a71a51d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:19.118546 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-dd932e77-24ef-48d5-88f1-7188523e23b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:19.190794 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 96/193] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:19.191592 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd932e77-24ef-48d5-88f1-7188523e23b8 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:19.210419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:19.233432 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2bc82-5e8d-4aaa-aa7d-46295a71a51d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:19.267405 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4ab43-7cef-4c19-8f72-95d36967dd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:19.268032 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4ab43-7cef-4c19-8f72-95d36967dd1d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:19.268311 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4ab43-7cef-4c19-8f72-95d36967dd1d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:19.268577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4ab43-7cef-4c19-8f72-95d36967dd1d None admin] PATH_INFO: `/v3/projects/793400b224a54914900adad496869b83` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:19.269707 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4ab43-7cef-4c19-8f72-95d36967dd1d None admin] RBAC: Authorizing `identity:get_project(project_id=793400b224a54914900adad496869b83)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:19.271170 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4ab43-7cef-4c19-8f72-95d36967dd1d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:19.277003 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 98/194] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:19 2022] GET /identity/v3/projects/793400b224a54914900adad496869b83 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:19.279051 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4ab43-7cef-4c19-8f72-95d36967dd1d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:19.304623 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd932e77-24ef-48d5-88f1-7188523e23b8 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:19.308314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:19.342921 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304d2d9-ef4b-4fc1-8560-3d0c7e775ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:19.343533 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304d2d9-ef4b-4fc1-8560-3d0c7e775ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:19.343810 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304d2d9-ef4b-4fc1-8560-3d0c7e775ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:19.344079 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304d2d9-ef4b-4fc1-8560-3d0c7e775ac5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:19.386545 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4ab43-7cef-4c19-8f72-95d36967dd1d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:19.416199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304d2d9-ef4b-4fc1-8560-3d0c7e775ac5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:19.417664 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304d2d9-ef4b-4fc1-8560-3d0c7e775ac5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:19.425503 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5d5c5-b914-4331-babc-62efd9fdc679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:19.426797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5d5c5-b914-4331-babc-62efd9fdc679 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:19.427253 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5d5c5-b914-4331-babc-62efd9fdc679 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:19.427695 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5d5c5-b914-4331-babc-62efd9fdc679 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:19.429774 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de5d5c5-b914-4331-babc-62efd9fdc679 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:19.432006 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de5d5c5-b914-4331-babc-62efd9fdc679 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:19.434475 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 99/195] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:54:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:19.435596 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5d5c5-b914-4331-babc-62efd9fdc679 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:19.456273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:19.544920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 97/196] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:54:19.544920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304d2d9-ef4b-4fc1-8560-3d0c7e775ac5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:19.545212 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5d5c5-b914-4331-babc-62efd9fdc679 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:19.545935 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:19.578996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13815087-3561-4cde-ac06-0079e9dc5df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:19.579606 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13815087-3561-4cde-ac06-0079e9dc5df0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:19.579878 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13815087-3561-4cde-ac06-0079e9dc5df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:19.580144 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13815087-3561-4cde-ac06-0079e9dc5df0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:19.580867 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13815087-3561-4cde-ac06-0079e9dc5df0 None admin] RBAC: Adding query filter params (role.id=7554b353873942fab506de830188aba0, scope.project.id=793400b224a54914900adad496869b83, user.id=166e77bb8b8f412f92ffdd1c1d42661f) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:19.581167 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13815087-3561-4cde-ac06-0079e9dc5df0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:19.582642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13815087-3561-4cde-ac06-0079e9dc5df0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:19.598793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 100/197] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:54:19 2022] GET /identity/v3/role_assignments?user.id=166e77bb8b8f412f92ffdd1c1d42661f&scope.project.id=793400b224a54914900adad496869b83&role.id=7554b353873942fab506de830188aba0 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:19.598793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13815087-3561-4cde-ac06-0079e9dc5df0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:19.667420 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304d2d9-ef4b-4fc1-8560-3d0c7e775ac5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:19.695431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:19.708756 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f42bd3-5daf-4586-bf5d-f8d7ab5a7f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:19.709291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f42bd3-5daf-4586-bf5d-f8d7ab5a7f6c None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:19.709569 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f42bd3-5daf-4586-bf5d-f8d7ab5a7f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:19.709838 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f42bd3-5daf-4586-bf5d-f8d7ab5a7f6c None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:19.710576 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f42bd3-5daf-4586-bf5d-f8d7ab5a7f6c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:19.712135 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f42bd3-5daf-4586-bf5d-f8d7ab5a7f6c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:19.716057 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13815087-3561-4cde-ac06-0079e9dc5df0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:19.753543 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017cbd54-71dc-4cc3-a593-692bab7605e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:19.754197 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cbd54-71dc-4cc3-a593-692bab7605e9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:19.754502 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cbd54-71dc-4cc3-a593-692bab7605e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:19.754771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cbd54-71dc-4cc3-a593-692bab7605e9 None admin] PATH_INFO: `/v3/projects/793400b224a54914900adad496869b83` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:19.755897 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017cbd54-71dc-4cc3-a593-692bab7605e9 None admin] RBAC: Authorizing `identity:get_project(project_id=793400b224a54914900adad496869b83)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:19.757331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017cbd54-71dc-4cc3-a593-692bab7605e9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:19.758991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 101/198] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:19 2022] GET /identity/v3/projects/793400b224a54914900adad496869b83 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:19.760249 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae049104-06a7-4e6a-81b5-4566c68e4e59 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:19.760533 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae049104-06a7-4e6a-81b5-4566c68e4e59 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:19.760796 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae049104-06a7-4e6a-81b5-4566c68e4e59 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:19.762004 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 102/199] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:19.765117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae049104-06a7-4e6a-81b5-4566c68e4e59 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:19.795196 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 98/200] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 10 16:54:19 2022] POST /identity/v3/users => generated 297 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:19.803565 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5142a-1596-459a-adb6-1f141d0cd704 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:19.803850 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5142a-1596-459a-adb6-1f141d0cd704 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:19.804130 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5142a-1596-459a-adb6-1f141d0cd704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:19.833744 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-52f5142a-1596-459a-adb6-1f141d0cd704 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:19.836213 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-52f5142a-1596-459a-adb6-1f141d0cd704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:19.904776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:19.905431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae049104-06a7-4e6a-81b5-4566c68e4e59 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:19.917747 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 99/201] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:19.942909 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5142a-1596-459a-adb6-1f141d0cd704 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:19.955858 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179f5d3-43ab-469b-92d0-d1768ca49212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:19.957576 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179f5d3-43ab-469b-92d0-d1768ca49212 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:19.958073 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179f5d3-43ab-469b-92d0-d1768ca49212 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:19.958693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179f5d3-43ab-469b-92d0-d1768ca49212 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:19.959961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9179f5d3-43ab-469b-92d0-d1768ca49212 None admin] RBAC: Adding query filter params (role.id=a3e5816caacc425c9a6dd381fe0eddf0, scope.project.id=793400b224a54914900adad496869b83, user.id=166e77bb8b8f412f92ffdd1c1d42661f) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:19.959961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9179f5d3-43ab-469b-92d0-d1768ca49212 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:19.962983 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9179f5d3-43ab-469b-92d0-d1768ca49212 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:19.971016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 103/202] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:54:19 2022] GET /identity/v3/role_assignments?user.id=166e77bb8b8f412f92ffdd1c1d42661f&scope.project.id=793400b224a54914900adad496869b83&role.id=a3e5816caacc425c9a6dd381fe0eddf0 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:20.073123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5142a-1596-459a-adb6-1f141d0cd704 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:20.112723 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f66741-0637-412f-97dc-1e2b55c7574e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:20.113456 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f66741-0637-412f-97dc-1e2b55c7574e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:20.113731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f66741-0637-412f-97dc-1e2b55c7574e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:20.114045 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f66741-0637-412f-97dc-1e2b55c7574e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:20.126097 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f66741-0637-412f-97dc-1e2b55c7574e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:20.130975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-27f66741-0637-412f-97dc-1e2b55c7574e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:54:20.130975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:20.130975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:20.130975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:20.130975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:20.130975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:20.130975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:20.130975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:20.130975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:20.130975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:20.130975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:20.130975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:20.130975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:20.130975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:20.130975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:20.130975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:20.131926 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:20.131926 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:20.131926 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:20.131926 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:20.131926 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:20.131926 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:20.131926 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:20.131926 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:20.131926 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:20.131926 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:20.131926 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:20.131926 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:20.131926 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:20.131926 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:20.131926 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:20.131926 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:20.132697 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:20.132697 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:20.132697 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:20.132697 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:20.132697 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:20.132697 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:20.132697 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:20.132697 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:20.132697 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:20.132697 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:20.132697 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:54:20.132697 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:54:20.136942 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 100/203] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:54:20.138365 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179f5d3-43ab-469b-92d0-d1768ca49212 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:20.159410 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:20.251126 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179f5d3-43ab-469b-92d0-d1768ca49212 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:20.288141 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59cdfd-435a-4282-b6b8-fa7ed1baf2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:20.288737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59cdfd-435a-4282-b6b8-fa7ed1baf2bc None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:20.289007 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59cdfd-435a-4282-b6b8-fa7ed1baf2bc None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:20.289271 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59cdfd-435a-4282-b6b8-fa7ed1baf2bc None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:20.289934 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59cdfd-435a-4282-b6b8-fa7ed1baf2bc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:20.290227 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59cdfd-435a-4282-b6b8-fa7ed1baf2bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:20.291679 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59cdfd-435a-4282-b6b8-fa7ed1baf2bc None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:20.300885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 104/204] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:20.306158 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f66741-0637-412f-97dc-1e2b55c7574e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:20.361541 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:20.415890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f66741-0637-412f-97dc-1e2b55c7574e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:20.452231 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda11e9e-ea6e-4488-bb69-33f1642d1c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:20.453055 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda11e9e-ea6e-4488-bb69-33f1642d1c88 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:20.453393 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda11e9e-ea6e-4488-bb69-33f1642d1c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:20.453726 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda11e9e-ea6e-4488-bb69-33f1642d1c88 None admin] PATH_INFO: `/v3/users/7e545de4333749148bc08c29a9785d7c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:20.454972 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda11e9e-ea6e-4488-bb69-33f1642d1c88 None admin] RBAC: Authorizing `identity:get_user(user_id=7e545de4333749148bc08c29a9785d7c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:20.456478 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda11e9e-ea6e-4488-bb69-33f1642d1c88 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:20.458451 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 101/205] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:20 2022] GET /identity/v3/users/7e545de4333749148bc08c29a9785d7c => generated 289 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:20.464666 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59cdfd-435a-4282-b6b8-fa7ed1baf2bc None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:20.564799 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:20.590770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59cdfd-435a-4282-b6b8-fa7ed1baf2bc None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:20.627301 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50966817-7419-49fd-af2d-52d75fab4e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:20.627954 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50966817-7419-49fd-af2d-52d75fab4e9a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:20.628232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50966817-7419-49fd-af2d-52d75fab4e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:20.628501 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50966817-7419-49fd-af2d-52d75fab4e9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:20.698269 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50966817-7419-49fd-af2d-52d75fab4e9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:20.699854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50966817-7419-49fd-af2d-52d75fab4e9a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:20.777747 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:20.803831 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 105/206] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:54:20.809730 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda11e9e-ea6e-4488-bb69-33f1642d1c88 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:20.961578 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda11e9e-ea6e-4488-bb69-33f1642d1c88 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:20.983410 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:21.002280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73156d62-2b0f-4ebe-bf94-35795eca52e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:21.003040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73156d62-2b0f-4ebe-bf94-35795eca52e6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:21.003316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73156d62-2b0f-4ebe-bf94-35795eca52e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:21.003585 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73156d62-2b0f-4ebe-bf94-35795eca52e6 None admin] PATH_INFO: `/v3/projects/1570ef44f3134332b1fc668345a37479` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:21.004820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73156d62-2b0f-4ebe-bf94-35795eca52e6 None admin] RBAC: Authorizing `identity:get_project(project_id=1570ef44f3134332b1fc668345a37479)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:21.006336 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73156d62-2b0f-4ebe-bf94-35795eca52e6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:21.008525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 102/207] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:20 2022] GET /identity/v3/projects/1570ef44f3134332b1fc668345a37479 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:21.015558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50966817-7419-49fd-af2d-52d75fab4e9a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:21.141311 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50966817-7419-49fd-af2d-52d75fab4e9a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:21.179534 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f352b78-66d7-4632-b2c1-df86bc2c72f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:21.180147 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f352b78-66d7-4632-b2c1-df86bc2c72f9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:21.180421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f352b78-66d7-4632-b2c1-df86bc2c72f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:21.180686 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f352b78-66d7-4632-b2c1-df86bc2c72f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:21.181409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f352b78-66d7-4632-b2c1-df86bc2c72f9 None admin] RBAC: Adding query filter params (role.id=dc46bbae6e0f42baaa61e88b45035c46, scope.project.id=1570ef44f3134332b1fc668345a37479, user.id=7e545de4333749148bc08c29a9785d7c) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:21.181710 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f352b78-66d7-4632-b2c1-df86bc2c72f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:21.183185 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f352b78-66d7-4632-b2c1-df86bc2c72f9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:21.190201 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 106/208] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:54:21 2022] GET /identity/v3/role_assignments?user.id=7e545de4333749148bc08c29a9785d7c&scope.project.id=1570ef44f3134332b1fc668345a37479&role.id=dc46bbae6e0f42baaa61e88b45035c46 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:21.973004 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194f409-9502-461e-8086-481f28b92455 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:21.973504 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194f409-9502-461e-8086-481f28b92455 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:21.973504 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194f409-9502-461e-8086-481f28b92455 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:21.975121 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 103/209] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:21.980236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa2581-5a3f-4125-bffe-0b3f82e6f7b3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:21.980519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa2581-5a3f-4125-bffe-0b3f82e6f7b3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:21.981333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa2581-5a3f-4125-bffe-0b3f82e6f7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:22.000839 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-82aa2581-5a3f-4125-bffe-0b3f82e6f7b3 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:22.002024 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-82aa2581-5a3f-4125-bffe-0b3f82e6f7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:22.033141 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:22.072847 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 107/210] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:22.095739 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194f409-9502-461e-8086-481f28b92455 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:22.177076 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397249b-cdf1-4748-8e3b-9e3808d3bbca None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:22.177393 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397249b-cdf1-4748-8e3b-9e3808d3bbca None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:22.177631 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397249b-cdf1-4748-8e3b-9e3808d3bbca None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:22.179302 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 108/211] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:22.184553 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb5ee4-1928-4305-a3b6-0a1ca3553f70 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:22.184977 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb5ee4-1928-4305-a3b6-0a1ca3553f70 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:22.185406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb5ee4-1928-4305-a3b6-0a1ca3553f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:22.204490 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4dbb5ee4-1928-4305-a3b6-0a1ca3553f70 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:22.205457 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4dbb5ee4-1928-4305-a3b6-0a1ca3553f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:22.208452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194f409-9502-461e-8086-481f28b92455 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:22.243162 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a24c95-79f0-4beb-8298-a9e23fd2ba36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:22.244079 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a24c95-79f0-4beb-8298-a9e23fd2ba36 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:22.244511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a24c95-79f0-4beb-8298-a9e23fd2ba36 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:22.244917 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a24c95-79f0-4beb-8298-a9e23fd2ba36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:22.256751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a24c95-79f0-4beb-8298-a9e23fd2ba36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:22.262823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-73a24c95-79f0-4beb-8298-a9e23fd2ba36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:54:22.262823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:22.262823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:22.262823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:22.262823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:22.262823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:22.262823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:22.262823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:22.262823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:22.262823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:22.262823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:22.262823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:22.262823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:22.262823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:22.262823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:22.262823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:22.263877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:22.263877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:22.263877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:22.263877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:22.263877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:22.263877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:22.263877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:22.263877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:22.263877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:22.263877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:22.263877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:22.263877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:22.263877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:22.263877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:22.263877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:22.263877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:22.264830 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:22.264830 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:22.264830 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:22.264830 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:22.264830 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:22.264830 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:22.264830 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:22.264830 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:22.264830 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:22.264830 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:22.264830 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:54:22.264830 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:54:22.267147 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 104/212] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:54:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:22.273985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a24c95-79f0-4beb-8298-a9e23fd2ba36 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:22.286494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 109/213] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:22.305257 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb5ee4-1928-4305-a3b6-0a1ca3553f70 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:22.324099 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:22.389239 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a24c95-79f0-4beb-8298-a9e23fd2ba36 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:22.428669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:22.432922 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb5ee4-1928-4305-a3b6-0a1ca3553f70 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:22.435475 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59f1da-726f-4ec3-a000-d265f0d1a0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:22.437024 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59f1da-726f-4ec3-a000-d265f0d1a0ab None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:22.437446 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59f1da-726f-4ec3-a000-d265f0d1a0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:22.437918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59f1da-726f-4ec3-a000-d265f0d1a0ab None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:22.438758 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed59f1da-726f-4ec3-a000-d265f0d1a0ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:22.439193 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed59f1da-726f-4ec3-a000-d265f0d1a0ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:22.440705 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed59f1da-726f-4ec3-a000-d265f0d1a0ab None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:22.453233 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 105/214] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:22.458693 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59f1da-726f-4ec3-a000-d265f0d1a0ab None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:22.473628 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de403dec-b733-441a-977a-4b3637f1b2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:22.474588 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de403dec-b733-441a-977a-4b3637f1b2d7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:22.475002 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de403dec-b733-441a-977a-4b3637f1b2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:22.475414 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de403dec-b733-441a-977a-4b3637f1b2d7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:22.486907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de403dec-b733-441a-977a-4b3637f1b2d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:22.491778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-de403dec-b733-441a-977a-4b3637f1b2d7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:54:22.491778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:22.491778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:22.491778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:22.491778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:22.491778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:22.491778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:22.491778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:22.491778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:22.491778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:22.491778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:22.491778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:22.491778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:22.491778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:22.491778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:22.491778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:22.492805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:22.492805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:22.492805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:22.492805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:22.492805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:22.492805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:22.492805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:22.492805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:22.492805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:22.492805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:22.492805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:22.492805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:22.492805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:22.492805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:22.492805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:22.492805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:22.493812 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:22.493812 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:22.493812 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:22.493812 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:22.493812 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:22.493812 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:22.493812 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:22.493812 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:22.493812 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:22.493812 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:22.493812 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:54:22.493812 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:22.496173 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 110/215] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 10 16:54:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:22.502775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de403dec-b733-441a-977a-4b3637f1b2d7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:22.572687 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59f1da-726f-4ec3-a000-d265f0d1a0ab None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:22.583543 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:22.610713 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de403dec-b733-441a-977a-4b3637f1b2d7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:22.614968 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2cda82-2fff-46f0-8557-e3403bf5e9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:22.614968 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cda82-2fff-46f0-8557-e3403bf5e9a4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:22.614968 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cda82-2fff-46f0-8557-e3403bf5e9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:22.614968 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cda82-2fff-46f0-8557-e3403bf5e9a4 None admin] PATH_INFO: `/v3/users/166e77bb8b8f412f92ffdd1c1d42661f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:22.616308 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2cda82-2fff-46f0-8557-e3403bf5e9a4 None admin] RBAC: Authorizing `identity:get_user(user_id=166e77bb8b8f412f92ffdd1c1d42661f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:22.618604 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2cda82-2fff-46f0-8557-e3403bf5e9a4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:22.625793 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 106/216] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:22 2022] GET /identity/v3/users/166e77bb8b8f412f92ffdd1c1d42661f => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:22.625793 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741dd80-78f1-4bfe-a027-ad363d806db2 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:22.625793 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741dd80-78f1-4bfe-a027-ad363d806db2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:22.625793 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741dd80-78f1-4bfe-a027-ad363d806db2 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:22.626533 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 107/217] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:22.627563 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642aec2-c725-4f36-9423-578d5fdc4773 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:22.627845 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642aec2-c725-4f36-9423-578d5fdc4773 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:22.628525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642aec2-c725-4f36-9423-578d5fdc4773 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:22.629840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 108/218] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:22.634092 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f642aec2-c725-4f36-9423-578d5fdc4773 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:22.651610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2dc9fe-bd9b-4ed6-9013-8af62aed8a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:22.652204 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2dc9fe-bd9b-4ed6-9013-8af62aed8a19 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:22.652481 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2dc9fe-bd9b-4ed6-9013-8af62aed8a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:22.652795 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2dc9fe-bd9b-4ed6-9013-8af62aed8a19 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:22.653540 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2dc9fe-bd9b-4ed6-9013-8af62aed8a19 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:22.653834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2dc9fe-bd9b-4ed6-9013-8af62aed8a19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:22.655293 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2dc9fe-bd9b-4ed6-9013-8af62aed8a19 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:22.663116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 111/219] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:54:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:22.664184 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606d762-a351-4751-85c4-db6732c244cb None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:22.664458 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606d762-a351-4751-85c4-db6732c244cb None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:22.664723 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606d762-a351-4751-85c4-db6732c244cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:22.681949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b606d762-a351-4751-85c4-db6732c244cb None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:22.683143 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b606d762-a351-4751-85c4-db6732c244cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:22.697231 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:22.738378 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f642aec2-c725-4f36-9423-578d5fdc4773 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:22.749999 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 112/220] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:22.751192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8a142-b0bc-464d-befc-d16c2e67c659 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:22.751477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8a142-b0bc-464d-befc-d16c2e67c659 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:22.751749 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8a142-b0bc-464d-befc-d16c2e67c659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:22.771856 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f13970-a67f-4e26-84fe-5999eda74351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:22.772649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f13970-a67f-4e26-84fe-5999eda74351 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:22.772922 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f13970-a67f-4e26-84fe-5999eda74351 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:22.773189 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f13970-a67f-4e26-84fe-5999eda74351 None admin] PATH_INFO: `/v3/projects/793400b224a54914900adad496869b83` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:22.774232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-32a8a142-b0bc-464d-befc-d16c2e67c659 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:22.774337 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f13970-a67f-4e26-84fe-5999eda74351 None admin] RBAC: Authorizing `identity:get_project(project_id=793400b224a54914900adad496869b83)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:22.775712 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-32a8a142-b0bc-464d-befc-d16c2e67c659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:22.776014 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f13970-a67f-4e26-84fe-5999eda74351 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:22.777937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 109/221] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:22 2022] GET /identity/v3/projects/793400b224a54914900adad496869b83 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:22.778690 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f13970-a67f-4e26-84fe-5999eda74351 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:22.825586 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:22.843279 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 113/222] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:22.851221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a8a142-b0bc-464d-befc-d16c2e67c659 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:22.883469 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f13970-a67f-4e26-84fe-5999eda74351 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:22.916171 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203f4b4-3e61-47de-8a87-f44e417b9870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:22.916796 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203f4b4-3e61-47de-8a87-f44e417b9870 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:22.917068 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203f4b4-3e61-47de-8a87-f44e417b9870 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:22.917334 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203f4b4-3e61-47de-8a87-f44e417b9870 None admin] PATH_INFO: `/v3/users/166e77bb8b8f412f92ffdd1c1d42661f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:22.919219 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203f4b4-3e61-47de-8a87-f44e417b9870 None admin] RBAC: Authorizing `identity:get_user(user_id=166e77bb8b8f412f92ffdd1c1d42661f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:22.920650 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203f4b4-3e61-47de-8a87-f44e417b9870 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:22.922215 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 110/223] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:22 2022] GET /identity/v3/users/166e77bb8b8f412f92ffdd1c1d42661f => generated 283 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:22.922968 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203f4b4-3e61-47de-8a87-f44e417b9870 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:22.931479 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:22.949828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a8a142-b0bc-464d-befc-d16c2e67c659 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:22.984477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ba8bf-35fe-4350-a93e-9ccc0c284e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:22.985097 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ba8bf-35fe-4350-a93e-9ccc0c284e22 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:22.985369 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ba8bf-35fe-4350-a93e-9ccc0c284e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:22.985632 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ba8bf-35fe-4350-a93e-9ccc0c284e22 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:22.996089 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35ba8bf-35fe-4350-a93e-9ccc0c284e22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:23.001456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-c35ba8bf-35fe-4350-a93e-9ccc0c284e22 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:23.001456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:23.001456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:23.001456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:23.001456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:23.001456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:23.001456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:23.001456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:23.001456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:23.001456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:23.001456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:23.001456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:23.001456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:23.001456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:23.001456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:23.001456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:23.002376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:23.002376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:23.002376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:23.002376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:23.002376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:23.002376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:23.002376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:23.002376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:23.002376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:23.002376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:23.002376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:23.002376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:23.002376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:23.002376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:23.002376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:23.002376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:23.003324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:23.003324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:23.003324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:23.003324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:23.003324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:23.003324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:23.003324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:23.003324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:23.003324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:23.003324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:23.003324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:23.003324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:23.003324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 114/224] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:23.003324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ba8bf-35fe-4350-a93e-9ccc0c284e22 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:23.028354 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203f4b4-3e61-47de-8a87-f44e417b9870 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:23.040704 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:23.061337 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebe44b-e12e-4552-abab-3104bca6ea85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:23.061985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebe44b-e12e-4552-abab-3104bca6ea85 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:23.062257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebe44b-e12e-4552-abab-3104bca6ea85 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:23.062547 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebe44b-e12e-4552-abab-3104bca6ea85 None admin] PATH_INFO: `/v3/projects/793400b224a54914900adad496869b83/users/166e77bb8b8f412f92ffdd1c1d42661f/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:23.064401 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ebe44b-e12e-4552-abab-3104bca6ea85 None admin] RBAC: Authorizing `identity:create_grant(project_id=793400b224a54914900adad496869b83, user_id=166e77bb8b8f412f92ffdd1c1d42661f, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:23.065888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ebe44b-e12e-4552-abab-3104bca6ea85 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:23.102486 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ba8bf-35fe-4350-a93e-9ccc0c284e22 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:23.168746 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 111/225] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 10 16:54:22 2022] PUT /identity/v3/projects/793400b224a54914900adad496869b83/users/166e77bb8b8f412f92ffdd1c1d42661f/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:23.169576 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebe44b-e12e-4552-abab-3104bca6ea85 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:23.198115 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25388e76-d982-4446-ba32-4b57ac04926d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:23.199302 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25388e76-d982-4446-ba32-4b57ac04926d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:23.200428 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25388e76-d982-4446-ba32-4b57ac04926d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:23.200711 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25388e76-d982-4446-ba32-4b57ac04926d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:23.202343 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25388e76-d982-4446-ba32-4b57ac04926d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:23.204598 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25388e76-d982-4446-ba32-4b57ac04926d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:23.212154 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 115/226] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:54:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:23.212154 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25388e76-d982-4446-ba32-4b57ac04926d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:23.227652 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:23.286117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebe44b-e12e-4552-abab-3104bca6ea85 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:23.329368 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579becb7-4a08-4588-b164-9472312695e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:23.330005 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579becb7-4a08-4588-b164-9472312695e7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:23.330287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579becb7-4a08-4588-b164-9472312695e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:23.330586 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579becb7-4a08-4588-b164-9472312695e7 None admin] PATH_INFO: `/v3/projects/793400b224a54914900adad496869b83` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:23.332357 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579becb7-4a08-4588-b164-9472312695e7 None admin] RBAC: Authorizing `identity:get_project(project_id=793400b224a54914900adad496869b83)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:23.333818 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579becb7-4a08-4588-b164-9472312695e7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:23.335514 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 112/227] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:23 2022] GET /identity/v3/projects/793400b224a54914900adad496869b83 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:23.336338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579becb7-4a08-4588-b164-9472312695e7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:23.337587 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25388e76-d982-4446-ba32-4b57ac04926d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:23.377251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c486d-24bd-4b58-b0f3-d26f0b9a0341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:23.377857 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c486d-24bd-4b58-b0f3-d26f0b9a0341 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:23.378128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c486d-24bd-4b58-b0f3-d26f0b9a0341 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:23.378417 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c486d-24bd-4b58-b0f3-d26f0b9a0341 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:23.379079 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c486d-24bd-4b58-b0f3-d26f0b9a0341 None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:23.379375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c486d-24bd-4b58-b0f3-d26f0b9a0341 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:23.380741 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c486d-24bd-4b58-b0f3-d26f0b9a0341 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:23.387887 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:23.389559 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 116/228] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:23.390530 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c486d-24bd-4b58-b0f3-d26f0b9a0341 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:23.454985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579becb7-4a08-4588-b164-9472312695e7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:23.474709 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:23.491163 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a01ee-0a48-4abf-85b0-c740eb4e3395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:23.491768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a01ee-0a48-4abf-85b0-c740eb4e3395 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:23.492049 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a01ee-0a48-4abf-85b0-c740eb4e3395 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:23.492323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a01ee-0a48-4abf-85b0-c740eb4e3395 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:23.492964 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a01ee-0a48-4abf-85b0-c740eb4e3395 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:23.494478 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a01ee-0a48-4abf-85b0-c740eb4e3395 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:23.506446 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c486d-24bd-4b58-b0f3-d26f0b9a0341 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:23.541222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d31c9c-8dbb-4d5c-a3a5-ff7ac5f03408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:23.541795 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d31c9c-8dbb-4d5c-a3a5-ff7ac5f03408 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:23.542068 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d31c9c-8dbb-4d5c-a3a5-ff7ac5f03408 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:23.542337 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d31c9c-8dbb-4d5c-a3a5-ff7ac5f03408 None admin] PATH_INFO: `/v3/projects/793400b224a54914900adad496869b83/users/166e77bb8b8f412f92ffdd1c1d42661f/roles/a3e5816caacc425c9a6dd381fe0eddf0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:23.545126 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d31c9c-8dbb-4d5c-a3a5-ff7ac5f03408 None admin] RBAC: Authorizing `identity:create_grant(project_id=793400b224a54914900adad496869b83, user_id=166e77bb8b8f412f92ffdd1c1d42661f, role_id=a3e5816caacc425c9a6dd381fe0eddf0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:23.546597 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d31c9c-8dbb-4d5c-a3a5-ff7ac5f03408 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:23.577143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 113/229] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 10 16:54:23 2022] POST /identity/v3/users => generated 297 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:23.578189 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a01ee-0a48-4abf-85b0-c740eb4e3395 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:23.631748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 117/230] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 10 16:54:23 2022] PUT /identity/v3/projects/793400b224a54914900adad496869b83/users/166e77bb8b8f412f92ffdd1c1d42661f/roles/a3e5816caacc425c9a6dd381fe0eddf0 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:23.719661 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:23.727488 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a01ee-0a48-4abf-85b0-c740eb4e3395 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:23.768966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73b28d-f5ae-495e-973a-6e214c954e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:23.769682 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73b28d-f5ae-495e-973a-6e214c954e0a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:23.769963 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73b28d-f5ae-495e-973a-6e214c954e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:23.770233 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73b28d-f5ae-495e-973a-6e214c954e0a None admin] PATH_INFO: `/v3/users/e9d44b03515c47a7956dc783e3195275` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:23.787835 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb73b28d-f5ae-495e-973a-6e214c954e0a None admin] RBAC: Authorizing `identity:get_user(user_id=e9d44b03515c47a7956dc783e3195275)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:23.789505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb73b28d-f5ae-495e-973a-6e214c954e0a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:23.791366 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 114/231] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:23 2022] GET /identity/v3/users/e9d44b03515c47a7956dc783e3195275 => generated 297 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:23.797253 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d31c9c-8dbb-4d5c-a3a5-ff7ac5f03408 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:23.916100 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d31c9c-8dbb-4d5c-a3a5-ff7ac5f03408 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:23.948445 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:23.955690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09421b3f-f8ef-49c0-9a19-1b0f5a21e681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:23.960458 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09421b3f-f8ef-49c0-9a19-1b0f5a21e681 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:23.960458 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09421b3f-f8ef-49c0-9a19-1b0f5a21e681 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:23.960458 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09421b3f-f8ef-49c0-9a19-1b0f5a21e681 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:24.037408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09421b3f-f8ef-49c0-9a19-1b0f5a21e681 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:24.038925 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09421b3f-f8ef-49c0-9a19-1b0f5a21e681 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:24.143891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 118/232] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:54:24.149480 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73b28d-f5ae-495e-973a-6e214c954e0a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:24.272001 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73b28d-f5ae-495e-973a-6e214c954e0a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:24.310447 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf187d-ad57-43bf-a2b3-236cfcdd22c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:24.311046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf187d-ad57-43bf-a2b3-236cfcdd22c6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:24.311318 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf187d-ad57-43bf-a2b3-236cfcdd22c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:24.311583 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf187d-ad57-43bf-a2b3-236cfcdd22c6 None admin] PATH_INFO: `/v3/projects/793400b224a54914900adad496869b83` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:24.312725 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bf187d-ad57-43bf-a2b3-236cfcdd22c6 None admin] RBAC: Authorizing `identity:get_project(project_id=793400b224a54914900adad496869b83)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:24.314169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bf187d-ad57-43bf-a2b3-236cfcdd22c6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:24.315850 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 115/233] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:24 2022] GET /identity/v3/projects/793400b224a54914900adad496869b83 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:24.321860 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09421b3f-f8ef-49c0-9a19-1b0f5a21e681 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:24.358510 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:24.446440 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09421b3f-f8ef-49c0-9a19-1b0f5a21e681 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:24.492446 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65791a-dbf8-428b-bc48-77e3feb2520c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:24.493051 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65791a-dbf8-428b-bc48-77e3feb2520c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:24.493324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65791a-dbf8-428b-bc48-77e3feb2520c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:24.493591 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65791a-dbf8-428b-bc48-77e3feb2520c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:24.494323 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65791a-dbf8-428b-bc48-77e3feb2520c None admin] RBAC: Adding query filter params (role.id=dfb72dc1f3744c8e957abcc4ac967dac, scope.project.id=793400b224a54914900adad496869b83, user.id=e9d44b03515c47a7956dc783e3195275) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:24.494651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65791a-dbf8-428b-bc48-77e3feb2520c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:24.496229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65791a-dbf8-428b-bc48-77e3feb2520c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:24.504686 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 119/234] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:54:24 2022] GET /identity/v3/role_assignments?user.id=e9d44b03515c47a7956dc783e3195275&scope.project.id=793400b224a54914900adad496869b83&role.id=dfb72dc1f3744c8e957abcc4ac967dac => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:25.442784 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4ddb1-22f1-48cf-91e7-0982e4fda425 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:25.443302 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4ddb1-22f1-48cf-91e7-0982e4fda425 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:25.443302 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4ddb1-22f1-48cf-91e7-0982e4fda425 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:25.444908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 116/235] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:25.450878 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325f4ad-10c9-4f91-8376-2d955a2b9b8e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:25.451273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325f4ad-10c9-4f91-8376-2d955a2b9b8e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:25.451457 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325f4ad-10c9-4f91-8376-2d955a2b9b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:25.479274 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0325f4ad-10c9-4f91-8376-2d955a2b9b8e None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:25.480478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0325f4ad-10c9-4f91-8376-2d955a2b9b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:25.561375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 120/236] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:25.585324 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f4ddb1-22f1-48cf-91e7-0982e4fda425 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:25.618922 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:25.715133 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f4ddb1-22f1-48cf-91e7-0982e4fda425 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:25.769516 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd342c-62b3-4295-b4bf-d45b9b179b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:25.770164 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd342c-62b3-4295-b4bf-d45b9b179b65 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:25.770490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd342c-62b3-4295-b4bf-d45b9b179b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:25.770769 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd342c-62b3-4295-b4bf-d45b9b179b65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:25.782607 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd342c-62b3-4295-b4bf-d45b9b179b65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:25.787373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-f7cd342c-62b3-4295-b4bf-d45b9b179b65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:54:25.787373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:25.787373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:25.787373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:25.787373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:25.787373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:25.787373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:25.787373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:25.787373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:25.787373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:25.787373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:25.787373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:25.787373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:25.787373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:25.787373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:25.787373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:25.788253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:25.788253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:25.788253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:25.788253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:25.788253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:25.788253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:25.788253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:25.788253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:25.788253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:25.788253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:25.788253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:25.788253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:25.788253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:25.788253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:25.788253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:25.788253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:25.789041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:25.789041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:25.789041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:25.789041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:25.789041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:25.789041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:25.789041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:25.789041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:25.789041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:25.789041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:25.789041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:54:25.789041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:54:25.789041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 117/237] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:54:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:25.795721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0325f4ad-10c9-4f91-8376-2d955a2b9b8e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:25.844178 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:25.933065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0325f4ad-10c9-4f91-8376-2d955a2b9b8e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:25.970218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6e6c6-ffcf-419c-9b5d-711146d26daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:25.971207 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6e6c6-ffcf-419c-9b5d-711146d26daa None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:25.971765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6e6c6-ffcf-419c-9b5d-711146d26daa None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:25.972231 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6e6c6-ffcf-419c-9b5d-711146d26daa None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:25.973099 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6e6c6-ffcf-419c-9b5d-711146d26daa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:25.973615 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6e6c6-ffcf-419c-9b5d-711146d26daa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:25.975209 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6e6c6-ffcf-419c-9b5d-711146d26daa None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:25.984400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 121/238] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:25.991435 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd342c-62b3-4295-b4bf-d45b9b179b65 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:26.115435 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:26.138847 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd342c-62b3-4295-b4bf-d45b9b179b65 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:26.180131 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d4b2e-f857-4a39-addb-f3e2e3580d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:26.180770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d4b2e-f857-4a39-addb-f3e2e3580d64 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:26.181051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d4b2e-f857-4a39-addb-f3e2e3580d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:26.181326 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d4b2e-f857-4a39-addb-f3e2e3580d64 None admin] PATH_INFO: `/v3/users/166e77bb8b8f412f92ffdd1c1d42661f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:26.182513 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d4b2e-f857-4a39-addb-f3e2e3580d64 None admin] RBAC: Authorizing `identity:get_user(user_id=166e77bb8b8f412f92ffdd1c1d42661f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:26.184535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d4b2e-f857-4a39-addb-f3e2e3580d64 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:26.186542 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 118/239] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:25 2022] GET /identity/v3/users/166e77bb8b8f412f92ffdd1c1d42661f => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:26.191888 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6e6c6-ffcf-419c-9b5d-711146d26daa None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:26.263179 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa0eb4-9f98-40a1-b94a-e3d7d55a1e7c None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:26.263364 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa0eb4-9f98-40a1-b94a-e3d7d55a1e7c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:26.263923 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa0eb4-9f98-40a1-b94a-e3d7d55a1e7c None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:26.265266 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 119/240] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:26.272747 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9e093-56e9-4557-b6af-6603da5b9bef None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:26.273034 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9e093-56e9-4557-b6af-6603da5b9bef None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:26.273309 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9e093-56e9-4557-b6af-6603da5b9bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:26.295782 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bfe9e093-56e9-4557-b6af-6603da5b9bef None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:26.297282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bfe9e093-56e9-4557-b6af-6603da5b9bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:26.341689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:26.369291 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6e6c6-ffcf-419c-9b5d-711146d26daa None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:26.415350 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 120/241] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:54:26.433505 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c95d21-85e5-4e9c-b96f-9a056e898cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:26.434784 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c95d21-85e5-4e9c-b96f-9a056e898cfb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:26.435062 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c95d21-85e5-4e9c-b96f-9a056e898cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:26.435328 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c95d21-85e5-4e9c-b96f-9a056e898cfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:26.471741 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe9e093-56e9-4557-b6af-6603da5b9bef None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:26.522176 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c95d21-85e5-4e9c-b96f-9a056e898cfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:26.523673 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c95d21-85e5-4e9c-b96f-9a056e898cfb None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:26.598845 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:26.622989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe9e093-56e9-4557-b6af-6603da5b9bef None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:26.626901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:26.647659 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 122/242] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:54:26.647659 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6b250-de28-41d6-9e93-4eb1684efa7d None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:26.647659 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6b250-de28-41d6-9e93-4eb1684efa7d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:26.647659 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6b250-de28-41d6-9e93-4eb1684efa7d None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:26.647659 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 123/243] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:26.647659 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af683f-2e48-46fd-95d2-01c866040561 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:26.648125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af683f-2e48-46fd-95d2-01c866040561 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:26.648538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af683f-2e48-46fd-95d2-01c866040561 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:26.649664 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 124/244] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:26.654387 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af683f-2e48-46fd-95d2-01c866040561 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:26.669366 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9922217-e691-4341-9238-336f321d9ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:26.670114 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9922217-e691-4341-9238-336f321d9ada None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:26.670409 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9922217-e691-4341-9238-336f321d9ada None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:26.670674 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9922217-e691-4341-9238-336f321d9ada None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:26.680895 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9922217-e691-4341-9238-336f321d9ada None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:26.686054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-b9922217-e691-4341-9238-336f321d9ada None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:54:26.686054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:26.686054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:26.686054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:26.686054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:26.686054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:26.686054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:26.686054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:26.686054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:26.686054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:26.686054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:26.686054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:26.686054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:26.686054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:26.686054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:26.686054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:26.686976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:26.686976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:26.686976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:26.686976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:26.686976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:26.686976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:26.686976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:26.686976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:26.686976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:26.686976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:26.686976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:26.686976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:26.686976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:26.686976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:26.686976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:26.686976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:26.687780 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:26.687780 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:26.687780 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:26.687780 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:26.687780 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:26.687780 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:26.687780 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:26.687780 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:26.687780 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:26.687780 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:26.687780 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:54:26.687780 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:54:26.687780 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 121/245] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 10 16:54:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:54:26.689023 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85153071-2249-40e5-96e2-ae651181faf3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:26.689023 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85153071-2249-40e5-96e2-ae651181faf3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:26.689023 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85153071-2249-40e5-96e2-ae651181faf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:26.708611 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-85153071-2249-40e5-96e2-ae651181faf3 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:26.709786 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-85153071-2249-40e5-96e2-ae651181faf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:26.763957 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af683f-2e48-46fd-95d2-01c866040561 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:26.798450 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 122/246] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:26.798450 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85e499-ce5d-4f81-9e14-710fc36e32a7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:26.798450 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85e499-ce5d-4f81-9e14-710fc36e32a7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:26.798450 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85e499-ce5d-4f81-9e14-710fc36e32a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:26.806075 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04327e3-e8f8-45ba-8207-7d9c3fcdb5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:26.806732 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04327e3-e8f8-45ba-8207-7d9c3fcdb5f7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:26.807004 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04327e3-e8f8-45ba-8207-7d9c3fcdb5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:26.807261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04327e3-e8f8-45ba-8207-7d9c3fcdb5f7 None admin] PATH_INFO: `/v3/projects/793400b224a54914900adad496869b83` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:26.808374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04327e3-e8f8-45ba-8207-7d9c3fcdb5f7 None admin] RBAC: Authorizing `identity:get_project(project_id=793400b224a54914900adad496869b83)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:26.810375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04327e3-e8f8-45ba-8207-7d9c3fcdb5f7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:26.815370 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-eb85e499-ce5d-4f81-9e14-710fc36e32a7 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:26.816818 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-eb85e499-ce5d-4f81-9e14-710fc36e32a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:26.816996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 125/247] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:26 2022] GET /identity/v3/projects/793400b224a54914900adad496869b83 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:26.817779 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04327e3-e8f8-45ba-8207-7d9c3fcdb5f7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:26.874423 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:26.883926 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:26.888444 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 123/248] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:26.889280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85e499-ce5d-4f81-9e14-710fc36e32a7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:26.922494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04327e3-e8f8-45ba-8207-7d9c3fcdb5f7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:26.956168 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df99746-fbda-415c-a00f-6ef26e40b10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:26.956765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df99746-fbda-415c-a00f-6ef26e40b10b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:26.957038 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df99746-fbda-415c-a00f-6ef26e40b10b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:26.957301 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df99746-fbda-415c-a00f-6ef26e40b10b None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:26.957954 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df99746-fbda-415c-a00f-6ef26e40b10b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:26.958248 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df99746-fbda-415c-a00f-6ef26e40b10b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:26.959639 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df99746-fbda-415c-a00f-6ef26e40b10b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:26.968223 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 126/249] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:54:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:26.969031 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df99746-fbda-415c-a00f-6ef26e40b10b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:26.991638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85e499-ce5d-4f81-9e14-710fc36e32a7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:27.025681 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ad0b2-a612-4a65-a06d-4d2f0f15c741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:27.026278 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ad0b2-a612-4a65-a06d-4d2f0f15c741 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:27.026667 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ad0b2-a612-4a65-a06d-4d2f0f15c741 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:27.026942 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ad0b2-a612-4a65-a06d-4d2f0f15c741 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:27.028250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365ad0b2-a612-4a65-a06d-4d2f0f15c741 None admin] RBAC: Adding query filter params (role.id=7554b353873942fab506de830188aba0, scope.project.id=793400b224a54914900adad496869b83, user.id=166e77bb8b8f412f92ffdd1c1d42661f) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:27.028552 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365ad0b2-a612-4a65-a06d-4d2f0f15c741 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:27.029988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365ad0b2-a612-4a65-a06d-4d2f0f15c741 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:27.035699 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 124/250] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:54:26 2022] GET /identity/v3/role_assignments?user.id=166e77bb8b8f412f92ffdd1c1d42661f&scope.project.id=793400b224a54914900adad496869b83&role.id=7554b353873942fab506de830188aba0 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:27.035699 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b194d7-b55b-4c9a-b260-d931406a7b79 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:27.035699 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b194d7-b55b-4c9a-b260-d931406a7b79 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:27.035699 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b194d7-b55b-4c9a-b260-d931406a7b79 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:27.035699 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 125/251] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:27.035699 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b194d7-b55b-4c9a-b260-d931406a7b79 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:27.077010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df99746-fbda-415c-a00f-6ef26e40b10b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:27.096734 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:27.114899 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:27.117645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf58039-26a0-46b0-913a-5216392ea626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:27.118545 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf58039-26a0-46b0-913a-5216392ea626 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:27.118999 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf58039-26a0-46b0-913a-5216392ea626 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:27.119444 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf58039-26a0-46b0-913a-5216392ea626 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:27.132010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf58039-26a0-46b0-913a-5216392ea626 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:27.139485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-0bf58039-26a0-46b0-913a-5216392ea626 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:54:27.139485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:27.139485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:27.139485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:27.139485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:27.139485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:27.139485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:27.139485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:27.139485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:27.139485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:27.139485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:27.139485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:27.139485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:27.139485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:27.139485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:27.139485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:27.140476 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:27.140476 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:27.140476 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:27.140476 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:27.140476 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:27.140476 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:27.140476 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:27.140476 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:27.140476 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:27.140476 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:27.140476 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:27.140476 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:27.140476 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:27.140476 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:27.140476 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:27.140476 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:27.147159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:27.147159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:27.147159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:27.147159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:27.147159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:27.147159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:27.147159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:27.147159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:27.147159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:27.147159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:27.147159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:54:27.147159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:27.147159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 127/252] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:27.147159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf58039-26a0-46b0-913a-5216392ea626 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:27.156777 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b194d7-b55b-4c9a-b260-d931406a7b79 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:27.195467 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae1b75-9a92-45c9-b1cb-50b0d9606be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:27.196185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae1b75-9a92-45c9-b1cb-50b0d9606be2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:27.196534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae1b75-9a92-45c9-b1cb-50b0d9606be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:27.196886 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae1b75-9a92-45c9-b1cb-50b0d9606be2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:27.198544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ae1b75-9a92-45c9-b1cb-50b0d9606be2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:27.200729 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ae1b75-9a92-45c9-b1cb-50b0d9606be2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:27.203589 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 126/253] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:54:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:27.203710 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e89135-3078-483b-b78d-9984b89e9bd0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:27.204002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e89135-3078-483b-b78d-9984b89e9bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:27.205385 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e89135-3078-483b-b78d-9984b89e9bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:27.224736 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-05e89135-3078-483b-b78d-9984b89e9bd0 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:27.226008 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-05e89135-3078-483b-b78d-9984b89e9bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:27.267511 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf58039-26a0-46b0-913a-5216392ea626 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:27.304943 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2dd88-7ad1-4705-8887-3480d6eb2a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:27.305564 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2dd88-7ad1-4705-8887-3480d6eb2a8e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:27.305834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2dd88-7ad1-4705-8887-3480d6eb2a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:27.306099 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2dd88-7ad1-4705-8887-3480d6eb2a8e None admin] PATH_INFO: `/v3/users/166e77bb8b8f412f92ffdd1c1d42661f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:27.307219 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d2dd88-7ad1-4705-8887-3480d6eb2a8e None admin] RBAC: Authorizing `identity:get_user(user_id=166e77bb8b8f412f92ffdd1c1d42661f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:27.308071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 127/254] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:27.309334 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e89135-3078-483b-b78d-9984b89e9bd0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:27.311051 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d2dd88-7ad1-4705-8887-3480d6eb2a8e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:27.312979 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 128/255] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:27 2022] GET /identity/v3/users/166e77bb8b8f412f92ffdd1c1d42661f => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:27.313477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2dd88-7ad1-4705-8887-3480d6eb2a8e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:27.359813 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:27.370658 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:27.408220 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e89135-3078-483b-b78d-9984b89e9bd0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:27.413454 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2dd88-7ad1-4705-8887-3480d6eb2a8e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:27.441505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e34005-d1d1-4eb9-a008-d5ca4ba09e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:27.442107 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e34005-d1d1-4eb9-a008-d5ca4ba09e9d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:27.442521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e34005-d1d1-4eb9-a008-d5ca4ba09e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:27.442795 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e34005-d1d1-4eb9-a008-d5ca4ba09e9d None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:27.443454 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e34005-d1d1-4eb9-a008-d5ca4ba09e9d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:27.443752 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e34005-d1d1-4eb9-a008-d5ca4ba09e9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:27.445122 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e34005-d1d1-4eb9-a008-d5ca4ba09e9d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:27.446438 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664931b9-055e-481a-831b-879de09d576f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:27.447043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664931b9-055e-481a-831b-879de09d576f None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:27.447324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664931b9-055e-481a-831b-879de09d576f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:27.447590 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664931b9-055e-481a-831b-879de09d576f None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:27.448224 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664931b9-055e-481a-831b-879de09d576f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:27.450184 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664931b9-055e-481a-831b-879de09d576f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:27.454042 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 128/256] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:27.454042 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e34005-d1d1-4eb9-a008-d5ca4ba09e9d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:27.540172 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 129/257] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 10 16:54:27 2022] POST /identity/v3/users => generated 293 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:27.541005 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664931b9-055e-481a-831b-879de09d576f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:27.554867 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e34005-d1d1-4eb9-a008-d5ca4ba09e9d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:27.583846 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:27.597469 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c87883-0c9c-46c6-abd8-36c46626aecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:27.598492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c87883-0c9c-46c6-abd8-36c46626aecd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:27.598925 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c87883-0c9c-46c6-abd8-36c46626aecd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:27.599372 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c87883-0c9c-46c6-abd8-36c46626aecd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:27.674552 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:27.678148 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664931b9-055e-481a-831b-879de09d576f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:27.684757 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c87883-0c9c-46c6-abd8-36c46626aecd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:27.688854 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c87883-0c9c-46c6-abd8-36c46626aecd None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:27.722661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d53c34-f2d0-4664-b0f4-90293b4e4efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:27.723303 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d53c34-f2d0-4664-b0f4-90293b4e4efe None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:27.723583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d53c34-f2d0-4664-b0f4-90293b4e4efe None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:27.723859 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d53c34-f2d0-4664-b0f4-90293b4e4efe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:27.737144 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d53c34-f2d0-4664-b0f4-90293b4e4efe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:27.742879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-87d53c34-f2d0-4664-b0f4-90293b4e4efe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:27.742879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:27.742879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:27.742879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:27.742879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:27.742879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:27.742879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:27.742879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:27.742879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:27.742879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:27.742879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:27.742879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:27.742879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:27.742879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:27.742879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:27.742879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:27.743949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:27.743949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:27.743949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:27.743949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:27.743949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:27.743949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:27.743949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:27.743949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:27.743949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:27.743949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:27.743949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:27.743949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:27.743949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:27.743949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:27.743949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:27.743949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:27.744967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:27.744967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:27.744967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:27.744967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:27.744967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:27.744967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:27.744967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:27.744967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:27.744967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:27.744967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:27.744967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:27.744967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:27.744967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 130/258] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:27.745969 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d53c34-f2d0-4664-b0f4-90293b4e4efe None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:27.816712 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 129/259] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:54:27.819396 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c87883-0c9c-46c6-abd8-36c46626aecd None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:27.846126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:27.877306 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d53c34-f2d0-4664-b0f4-90293b4e4efe None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:27.924904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f375c1-a93b-47df-b29f-ed1196a7df84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:27.925536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f375c1-a93b-47df-b29f-ed1196a7df84 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:27.925814 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f375c1-a93b-47df-b29f-ed1196a7df84 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:27.926084 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f375c1-a93b-47df-b29f-ed1196a7df84 None admin] PATH_INFO: `/v3/users/e5c97ea444ec49788b6a9bd381886011` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:27.943260 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f375c1-a93b-47df-b29f-ed1196a7df84 None admin] RBAC: Authorizing `identity:get_user(user_id=e5c97ea444ec49788b6a9bd381886011)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:27.945038 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f375c1-a93b-47df-b29f-ed1196a7df84 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:27.945849 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c87883-0c9c-46c6-abd8-36c46626aecd None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:27.947061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 131/260] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:27 2022] GET /identity/v3/users/e5c97ea444ec49788b6a9bd381886011 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:27.948123 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f375c1-a93b-47df-b29f-ed1196a7df84 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:27.972132 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:27.986241 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f5496-3e8b-4b7b-b71c-9b90c290d75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:27.986894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f5496-3e8b-4b7b-b71c-9b90c290d75c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:27.987172 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f5496-3e8b-4b7b-b71c-9b90c290d75c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:27.988199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f5496-3e8b-4b7b-b71c-9b90c290d75c None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:27.988199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3f5496-3e8b-4b7b-b71c-9b90c290d75c None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:27.988365 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3f5496-3e8b-4b7b-b71c-9b90c290d75c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:27.989763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3f5496-3e8b-4b7b-b71c-9b90c290d75c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:27.999517 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 130/261] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:28.000366 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f5496-3e8b-4b7b-b71c-9b90c290d75c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:28.080985 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f375c1-a93b-47df-b29f-ed1196a7df84 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:28.104495 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:28.120249 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f5496-3e8b-4b7b-b71c-9b90c290d75c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:28.128696 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616fb50-b2ab-407e-bc0b-b0957d0a0a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:28.129345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616fb50-b2ab-407e-bc0b-b0957d0a0a44 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:28.130581 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616fb50-b2ab-407e-bc0b-b0957d0a0a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:28.130927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616fb50-b2ab-407e-bc0b-b0957d0a0a44 None admin] PATH_INFO: `/v3/projects/793400b224a54914900adad496869b83` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:28.133409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2616fb50-b2ab-407e-bc0b-b0957d0a0a44 None admin] RBAC: Authorizing `identity:get_project(project_id=793400b224a54914900adad496869b83)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:28.135241 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2616fb50-b2ab-407e-bc0b-b0957d0a0a44 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:28.137795 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 132/262] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:27 2022] GET /identity/v3/projects/793400b224a54914900adad496869b83 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:28.143328 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616fb50-b2ab-407e-bc0b-b0957d0a0a44 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:28.163324 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca64fbb3-de2b-4c6b-8f19-887718143779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:28.164081 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64fbb3-de2b-4c6b-8f19-887718143779 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:28.164454 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64fbb3-de2b-4c6b-8f19-887718143779 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:28.164831 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64fbb3-de2b-4c6b-8f19-887718143779 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:28.237948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca64fbb3-de2b-4c6b-8f19-887718143779 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:28.240109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca64fbb3-de2b-4c6b-8f19-887718143779 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:28.257909 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:28.273338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616fb50-b2ab-407e-bc0b-b0957d0a0a44 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:28.315890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d1a59-b4d1-4308-ad22-b20d22e99b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:28.317090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d1a59-b4d1-4308-ad22-b20d22e99b11 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:28.317507 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d1a59-b4d1-4308-ad22-b20d22e99b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:28.318045 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d1a59-b4d1-4308-ad22-b20d22e99b11 None admin] PATH_INFO: `/v3/users/e9d44b03515c47a7956dc783e3195275` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:28.319955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d1a59-b4d1-4308-ad22-b20d22e99b11 None admin] RBAC: Authorizing `identity:get_user(user_id=e9d44b03515c47a7956dc783e3195275)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:28.321571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d1a59-b4d1-4308-ad22-b20d22e99b11 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:28.324028 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 133/263] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:28 2022] GET /identity/v3/users/e9d44b03515c47a7956dc783e3195275 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:28.329039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d1a59-b4d1-4308-ad22-b20d22e99b11 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:28.354796 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:28.365113 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 131/264] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:54:28.365880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca64fbb3-de2b-4c6b-8f19-887718143779 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:28.441683 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d1a59-b4d1-4308-ad22-b20d22e99b11 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:28.469747 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca64fbb3-de2b-4c6b-8f19-887718143779 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:28.479891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e0bf0-3a79-4df5-a528-f1cb2fe3ec53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:28.480718 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e0bf0-3a79-4df5-a528-f1cb2fe3ec53 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:28.481262 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e0bf0-3a79-4df5-a528-f1cb2fe3ec53 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:28.481684 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e0bf0-3a79-4df5-a528-f1cb2fe3ec53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:28.482710 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e0bf0-3a79-4df5-a528-f1cb2fe3ec53 None admin] RBAC: Adding query filter params (role.id=a3e5816caacc425c9a6dd381fe0eddf0, scope.project.id=793400b224a54914900adad496869b83, user.id=166e77bb8b8f412f92ffdd1c1d42661f) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:28.483159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e0bf0-3a79-4df5-a528-f1cb2fe3ec53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:28.484830 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e0bf0-3a79-4df5-a528-f1cb2fe3ec53 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:28.498855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 134/265] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:54:28 2022] GET /identity/v3/role_assignments?user.id=166e77bb8b8f412f92ffdd1c1d42661f&scope.project.id=793400b224a54914900adad496869b83&role.id=a3e5816caacc425c9a6dd381fe0eddf0 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:28.500079 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e0bf0-3a79-4df5-a528-f1cb2fe3ec53 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:28.507117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67ee68-bb28-4456-b583-3185f9e48f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:28.507869 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67ee68-bb28-4456-b583-3185f9e48f2c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:28.508267 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67ee68-bb28-4456-b583-3185f9e48f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:28.508641 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67ee68-bb28-4456-b583-3185f9e48f2c None admin] PATH_INFO: `/v3/projects/793400b224a54914900adad496869b83` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:28.516349 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67ee68-bb28-4456-b583-3185f9e48f2c None admin] RBAC: Authorizing `identity:get_project(project_id=793400b224a54914900adad496869b83)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:28.518011 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67ee68-bb28-4456-b583-3185f9e48f2c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:28.524182 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 132/266] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:28 2022] GET /identity/v3/projects/793400b224a54914900adad496869b83 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:28.532348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67ee68-bb28-4456-b583-3185f9e48f2c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:28.544186 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:28.672534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:28.710387 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e0bf0-3a79-4df5-a528-f1cb2fe3ec53 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:28.715927 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67ee68-bb28-4456-b583-3185f9e48f2c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:28.748302 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a7111-3c9d-4614-b448-e1587c0191b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:28.749371 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a7111-3c9d-4614-b448-e1587c0191b1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:28.749860 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a7111-3c9d-4614-b448-e1587c0191b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:28.750501 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a7111-3c9d-4614-b448-e1587c0191b1 None admin] PATH_INFO: `/v3/projects/793400b224a54914900adad496869b83` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:28.752034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a7111-3c9d-4614-b448-e1587c0191b1 None admin] RBAC: Authorizing `identity:get_project(project_id=793400b224a54914900adad496869b83)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:28.754064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a7111-3c9d-4614-b448-e1587c0191b1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:28.754258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa272a-4cd2-4cfe-a37d-301f7b430ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:28.754963 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa272a-4cd2-4cfe-a37d-301f7b430ea1 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:28.755240 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa272a-4cd2-4cfe-a37d-301f7b430ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:28.755511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa272a-4cd2-4cfe-a37d-301f7b430ea1 None admin] PATH_INFO: `/v3/projects/793400b224a54914900adad496869b83/users/e9d44b03515c47a7956dc783e3195275/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:28.757257 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 135/267] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:28 2022] GET /identity/v3/projects/793400b224a54914900adad496869b83 => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:28.757445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa272a-4cd2-4cfe-a37d-301f7b430ea1 None admin] RBAC: Authorizing `identity:create_grant(project_id=793400b224a54914900adad496869b83, user_id=e9d44b03515c47a7956dc783e3195275, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:28.761775 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa272a-4cd2-4cfe-a37d-301f7b430ea1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:28.763792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a7111-3c9d-4614-b448-e1587c0191b1 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:28.842194 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 133/268] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 10 16:54:28 2022] PUT /identity/v3/projects/793400b224a54914900adad496869b83/users/e9d44b03515c47a7956dc783e3195275/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:28.871702 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:28.890880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a7111-3c9d-4614-b448-e1587c0191b1 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:28.934406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f7c15-a10a-4a88-adcf-d499ef70bb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:28.935042 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f7c15-a10a-4a88-adcf-d499ef70bb58 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:28.935356 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f7c15-a10a-4a88-adcf-d499ef70bb58 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:28.935624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f7c15-a10a-4a88-adcf-d499ef70bb58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:28.936369 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569f7c15-a10a-4a88-adcf-d499ef70bb58 None admin] RBAC: Adding query filter params (role.id=dc46bbae6e0f42baaa61e88b45035c46, scope.project.id=793400b224a54914900adad496869b83, user.id=e5c97ea444ec49788b6a9bd381886011) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:28.936675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569f7c15-a10a-4a88-adcf-d499ef70bb58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:28.938216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569f7c15-a10a-4a88-adcf-d499ef70bb58 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:28.944781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 136/269] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:54:28 2022] GET /identity/v3/role_assignments?user.id=e5c97ea444ec49788b6a9bd381886011&scope.project.id=793400b224a54914900adad496869b83&role.id=dc46bbae6e0f42baaa61e88b45035c46 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:29.740064 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b0ddc-e320-4c8d-8b35-aa3554cf19a9 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:29.740568 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b0ddc-e320-4c8d-8b35-aa3554cf19a9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:29.740568 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b0ddc-e320-4c8d-8b35-aa3554cf19a9 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:29.742078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 134/270] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:29.747603 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55676710-84f1-451f-a0e3-5b9aab8d0e59 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:29.747999 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55676710-84f1-451f-a0e3-5b9aab8d0e59 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:29.748182 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55676710-84f1-451f-a0e3-5b9aab8d0e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:29.766565 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-55676710-84f1-451f-a0e3-5b9aab8d0e59 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:29.767738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-55676710-84f1-451f-a0e3-5b9aab8d0e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:29.840563 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 137/271] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:29.863032 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b0ddc-e320-4c8d-8b35-aa3554cf19a9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:29.889024 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7bc1fc-12ff-4f11-bb3d-cddf4e7f7b4c None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:29.889306 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7bc1fc-12ff-4f11-bb3d-cddf4e7f7b4c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:29.889579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7bc1fc-12ff-4f11-bb3d-cddf4e7f7b4c None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:29.891243 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 138/272] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:54:29.896100 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1286f-86fc-48cc-942a-7175f56e0474 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:29.896386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1286f-86fc-48cc-942a-7175f56e0474 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:29.896659 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1286f-86fc-48cc-942a-7175f56e0474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:29.916102 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2af1286f-86fc-48cc-942a-7175f56e0474 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:29.917279 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2af1286f-86fc-48cc-942a-7175f56e0474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:29.978321 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b0ddc-e320-4c8d-8b35-aa3554cf19a9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:29.987557 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 139/273] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:30.012120 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:30.014298 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af1286f-86fc-48cc-942a-7175f56e0474 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:30.015941 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7467df-db9f-42fa-9131-2f7021e9e41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:30.016659 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7467df-db9f-42fa-9131-2f7021e9e41c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:30.017009 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7467df-db9f-42fa-9131-2f7021e9e41c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:30.017500 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7467df-db9f-42fa-9131-2f7021e9e41c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:30.028374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:30.030384 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7467df-db9f-42fa-9131-2f7021e9e41c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:30.036536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-eb7467df-db9f-42fa-9131-2f7021e9e41c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:30.036536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:30.036536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:30.036536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:30.036536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:30.036536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:30.036536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:30.036536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:30.036536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:30.036536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:30.036536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:30.036536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:30.036536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:30.036536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:30.036536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:30.036536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:30.037498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:30.037498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:30.037498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:30.037498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:30.037498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:30.037498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:30.037498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:30.037498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:30.037498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:30.037498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:30.037498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:30.037498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:30.037498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:30.037498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:30.037498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:30.037498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:30.038505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:30.038505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:30.038505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:30.038505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:30.038505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:30.038505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:30.038505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:30.038505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:30.038505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:30.038505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:30.038505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:30.038505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:54:30.038505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 135/274] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:30.044934 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7467df-db9f-42fa-9131-2f7021e9e41c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:30.133697 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af1286f-86fc-48cc-942a-7175f56e0474 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:30.222136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4275ad-64db-425e-a116-21ff4763837a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:30.223386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4275ad-64db-425e-a116-21ff4763837a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:30.223889 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4275ad-64db-425e-a116-21ff4763837a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:30.224739 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4275ad-64db-425e-a116-21ff4763837a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:30.226155 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4275ad-64db-425e-a116-21ff4763837a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:30.228595 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4275ad-64db-425e-a116-21ff4763837a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:30.235912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 140/275] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:54:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:30.238264 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4275ad-64db-425e-a116-21ff4763837a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:30.244394 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7467df-db9f-42fa-9131-2f7021e9e41c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:30.280564 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e124a-b4e1-4919-a067-a9e38bed585b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:30.281319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e124a-b4e1-4919-a067-a9e38bed585b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:30.281704 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e124a-b4e1-4919-a067-a9e38bed585b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:30.282074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e124a-b4e1-4919-a067-a9e38bed585b None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:30.282889 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e124a-b4e1-4919-a067-a9e38bed585b None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:30.283289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e124a-b4e1-4919-a067-a9e38bed585b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:30.284774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e124a-b4e1-4919-a067-a9e38bed585b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:30.298144 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 136/276] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:30.299167 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e124a-b4e1-4919-a067-a9e38bed585b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:30.328059 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:30.335328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:30.352506 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4275ad-64db-425e-a116-21ff4763837a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:30.400608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c5461-df3c-4729-822b-fd4e3de70cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:30.401412 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c5461-df3c-4729-822b-fd4e3de70cd1 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:30.401697 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c5461-df3c-4729-822b-fd4e3de70cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:30.401970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c5461-df3c-4729-822b-fd4e3de70cd1 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:30.402680 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c5461-df3c-4729-822b-fd4e3de70cd1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:30.404651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c5461-df3c-4729-822b-fd4e3de70cd1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:30.410206 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e124a-b4e1-4919-a067-a9e38bed585b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:30.446269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91000749-660e-4d3d-a190-e5b9c3c552e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:30.447164 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91000749-660e-4d3d-a190-e5b9c3c552e5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:30.447640 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91000749-660e-4d3d-a190-e5b9c3c552e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:30.448048 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91000749-660e-4d3d-a190-e5b9c3c552e5 None admin] PATH_INFO: `/v3/users/d166bede7100477b91d4a9ee24b4100f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:30.462622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91000749-660e-4d3d-a190-e5b9c3c552e5 None admin] RBAC: Authorizing `identity:get_user(user_id=d166bede7100477b91d4a9ee24b4100f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:30.464324 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91000749-660e-4d3d-a190-e5b9c3c552e5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:30.466868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 137/277] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:30 2022] GET /identity/v3/users/d166bede7100477b91d4a9ee24b4100f => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:30.472251 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91000749-660e-4d3d-a190-e5b9c3c552e5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:30.515625 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 141/278] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 10 16:54:30 2022] POST /identity/v3/users => generated 293 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:54:30.590614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:30.599295 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91000749-660e-4d3d-a190-e5b9c3c552e5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:30.639857 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ee973-1e1c-49fd-a4e6-c3159a3c9183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:30.640539 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ee973-1e1c-49fd-a4e6-c3159a3c9183 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:30.640829 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ee973-1e1c-49fd-a4e6-c3159a3c9183 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:30.641105 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ee973-1e1c-49fd-a4e6-c3159a3c9183 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:30.641849 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ee973-1e1c-49fd-a4e6-c3159a3c9183 None admin] RBAC: Adding query filter params (role.id=dfb72dc1f3744c8e957abcc4ac967dac, user.id=d166bede7100477b91d4a9ee24b4100f, scope.system=all) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:30.642158 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ee973-1e1c-49fd-a4e6-c3159a3c9183 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:30.643765 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ee973-1e1c-49fd-a4e6-c3159a3c9183 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:30.650387 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 138/279] 10.222.0.45 () {56 vars in 1308 bytes} [Wed Aug 10 16:54:30 2022] GET /identity/v3/role_assignments?user.id=d166bede7100477b91d4a9ee24b4100f&scope.system=all&role.id=dfb72dc1f3744c8e957abcc4ac967dac => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:30.970846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb98e4-f80f-47d4-b74d-1ee4fae84ac7 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:30.971392 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb98e4-f80f-47d4-b74d-1ee4fae84ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:30.971392 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb98e4-f80f-47d4-b74d-1ee4fae84ac7 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:30.972737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 142/280] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:30.978122 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5df730-4823-4356-b10a-23d41df7c28f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:30.978578 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5df730-4823-4356-b10a-23d41df7c28f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:30.978720 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5df730-4823-4356-b10a-23d41df7c28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:30.999162 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6a5df730-4823-4356-b10a-23d41df7c28f None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:31.000626 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6a5df730-4823-4356-b10a-23d41df7c28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:31.022394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbcc792-ea32-40ea-9a24-85429404111f None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:31.022692 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbcc792-ea32-40ea-9a24-85429404111f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:31.022970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbcc792-ea32-40ea-9a24-85429404111f None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:31.024599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 143/281] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:31.030988 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39309fe4-d6ea-49bc-9b76-fa8e17c0fc43 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:31.031264 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39309fe4-d6ea-49bc-9b76-fa8e17c0fc43 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:31.031535 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39309fe4-d6ea-49bc-9b76-fa8e17c0fc43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:31.051583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-39309fe4-d6ea-49bc-9b76-fa8e17c0fc43 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:31.052847 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-39309fe4-d6ea-49bc-9b76-fa8e17c0fc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:31.109690 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 139/282] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:31.135655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5df730-4823-4356-b10a-23d41df7c28f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:31.213882 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 144/283] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:54:31.268849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39309fe4-d6ea-49bc-9b76-fa8e17c0fc43 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:31.283853 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:31.296439 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5df730-4823-4356-b10a-23d41df7c28f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:31.339057 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e1b19-7762-406e-abea-91a5bb71e8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:31.339685 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e1b19-7762-406e-abea-91a5bb71e8a1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:31.340064 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e1b19-7762-406e-abea-91a5bb71e8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:31.340397 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e1b19-7762-406e-abea-91a5bb71e8a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:31.361581 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e1b19-7762-406e-abea-91a5bb71e8a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:31.366622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-738e1b19-7762-406e-abea-91a5bb71e8a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:31.366622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:31.366622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:31.366622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:31.366622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:31.366622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:31.366622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:31.366622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:31.366622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:31.366622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:31.366622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:31.366622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:31.366622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:31.366622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:31.366622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:31.366622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:31.367508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:31.367508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:31.367508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:31.367508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:31.367508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:31.367508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:31.367508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:31.367508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:31.367508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:31.367508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:31.367508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:31.367508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:31.367508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:31.367508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:31.367508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:31.367508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:31.368308 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:31.368308 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:31.368308 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:31.368308 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:31.368308 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:31.368308 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:31.368308 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:31.368308 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:31.368308 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:31.368308 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:31.368308 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:31.368308 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:54:31.368308 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 140/284] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:54:31.375918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e1b19-7762-406e-abea-91a5bb71e8a1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:31.379794 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:31.456888 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39309fe4-d6ea-49bc-9b76-fa8e17c0fc43 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:31.484535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e1b19-7762-406e-abea-91a5bb71e8a1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:31.520707 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf91513-8957-4fbe-b4ed-3c264f942940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:31.521557 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf91513-8957-4fbe-b4ed-3c264f942940 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:31.521991 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf91513-8957-4fbe-b4ed-3c264f942940 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:31.522435 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf91513-8957-4fbe-b4ed-3c264f942940 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:31.523318 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf91513-8957-4fbe-b4ed-3c264f942940 None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:31.523783 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf91513-8957-4fbe-b4ed-3c264f942940 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:31.525559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf91513-8957-4fbe-b4ed-3c264f942940 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:31.527063 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69c569-c356-479a-868f-0eb88396d35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:31.527754 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69c569-c356-479a-868f-0eb88396d35c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:31.527995 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69c569-c356-479a-868f-0eb88396d35c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:31.528266 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69c569-c356-479a-868f-0eb88396d35c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:31.535343 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 141/285] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:31.541219 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf91513-8957-4fbe-b4ed-3c264f942940 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:31.544364 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c69c569-c356-479a-868f-0eb88396d35c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:31.554713 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:31.557375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-0c69c569-c356-479a-868f-0eb88396d35c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:54:31.557375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:31.557375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:31.557375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:31.557375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:31.557375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:31.557375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:31.557375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:31.557375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:31.557375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:31.557375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:31.557375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:31.557375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:31.557375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:31.557375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:31.557375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:31.558481 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:31.558481 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:31.558481 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:31.558481 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:31.558481 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:31.558481 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:31.558481 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:31.558481 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:31.558481 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:31.558481 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:31.558481 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:31.558481 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:31.558481 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:31.558481 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:31.558481 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:31.558481 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:31.560210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:31.560210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:31.560210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:31.560210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:31.560210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:31.560210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:31.560210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:31.560210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:31.560210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:31.560210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:31.560210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:54:31.560210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:31.560210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 145/286] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:54:31.564855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69c569-c356-479a-868f-0eb88396d35c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:31.671625 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf91513-8957-4fbe-b4ed-3c264f942940 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:31.707238 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2391fd-10a7-4d9e-a5ba-1f40c5f5aa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:31.708354 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2391fd-10a7-4d9e-a5ba-1f40c5f5aa20 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:31.708831 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2391fd-10a7-4d9e-a5ba-1f40c5f5aa20 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:31.709301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2391fd-10a7-4d9e-a5ba-1f40c5f5aa20 None admin] PATH_INFO: `/v3/users/e9d44b03515c47a7956dc783e3195275` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:31.710779 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2391fd-10a7-4d9e-a5ba-1f40c5f5aa20 None admin] RBAC: Authorizing `identity:get_user(user_id=e9d44b03515c47a7956dc783e3195275)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:31.712565 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2391fd-10a7-4d9e-a5ba-1f40c5f5aa20 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:31.719071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 142/287] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:31 2022] GET /identity/v3/users/e9d44b03515c47a7956dc783e3195275 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:31.720912 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2391fd-10a7-4d9e-a5ba-1f40c5f5aa20 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:31.732478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69c569-c356-479a-868f-0eb88396d35c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:31.757675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:31.787135 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a443a-6b0e-4d72-9762-6f4a9a176431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:31.787670 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a443a-6b0e-4d72-9762-6f4a9a176431 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:31.787951 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a443a-6b0e-4d72-9762-6f4a9a176431 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:31.788224 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a443a-6b0e-4d72-9762-6f4a9a176431 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:31.788897 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a443a-6b0e-4d72-9762-6f4a9a176431 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:31.789200 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a443a-6b0e-4d72-9762-6f4a9a176431 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:31.790764 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a443a-6b0e-4d72-9762-6f4a9a176431 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:31.806065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 146/288] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:31.810774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:31.815369 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a443a-6b0e-4d72-9762-6f4a9a176431 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:31.843014 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2391fd-10a7-4d9e-a5ba-1f40c5f5aa20 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:31.900579 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a35b88-3bda-4328-be98-0fc616939d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:31.901296 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a35b88-3bda-4328-be98-0fc616939d13 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:31.901577 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a35b88-3bda-4328-be98-0fc616939d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:31.901847 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a35b88-3bda-4328-be98-0fc616939d13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:31.920633 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a443a-6b0e-4d72-9762-6f4a9a176431 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:31.956927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edde869-897a-4a73-baff-00b4d339eb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:31.957559 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edde869-897a-4a73-baff-00b4d339eb11 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:31.957835 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edde869-897a-4a73-baff-00b4d339eb11 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:31.958103 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edde869-897a-4a73-baff-00b4d339eb11 None admin] PATH_INFO: `/v3/users/e5c97ea444ec49788b6a9bd381886011` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:31.959275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edde869-897a-4a73-baff-00b4d339eb11 None admin] RBAC: Authorizing `identity:get_user(user_id=e5c97ea444ec49788b6a9bd381886011)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:31.962402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edde869-897a-4a73-baff-00b4d339eb11 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:31.967624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 147/289] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:31 2022] GET /identity/v3/users/e5c97ea444ec49788b6a9bd381886011 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:31.968500 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edde869-897a-4a73-baff-00b4d339eb11 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:32.037355 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:32.073059 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edde869-897a-4a73-baff-00b4d339eb11 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:32.091409 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a35b88-3bda-4328-be98-0fc616939d13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:32.093827 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a35b88-3bda-4328-be98-0fc616939d13 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:32.109415 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5af12-9b37-486e-8f78-47bfbfe04edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:32.110043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5af12-9b37-486e-8f78-47bfbfe04edf None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:32.110467 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5af12-9b37-486e-8f78-47bfbfe04edf None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:32.110746 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5af12-9b37-486e-8f78-47bfbfe04edf None admin] PATH_INFO: `/v3/projects/793400b224a54914900adad496869b83` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:32.111897 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5af12-9b37-486e-8f78-47bfbfe04edf None admin] RBAC: Authorizing `identity:get_project(project_id=793400b224a54914900adad496869b83)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:32.113351 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5af12-9b37-486e-8f78-47bfbfe04edf None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:32.115485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 148/290] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:31 2022] GET /identity/v3/projects/793400b224a54914900adad496869b83 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:32.121227 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5af12-9b37-486e-8f78-47bfbfe04edf None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:32.225821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5af12-9b37-486e-8f78-47bfbfe04edf None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:32.250966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:32.262263 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bf205-87c4-4e35-b7e5-a296ac56d570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:32.262978 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bf205-87c4-4e35-b7e5-a296ac56d570 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:32.263237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bf205-87c4-4e35-b7e5-a296ac56d570 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:32.267236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bf205-87c4-4e35-b7e5-a296ac56d570 None admin] PATH_INFO: `/v3/projects/793400b224a54914900adad496869b83/users/e5c97ea444ec49788b6a9bd381886011/roles/dc46bbae6e0f42baaa61e88b45035c46` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:32.267236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940bf205-87c4-4e35-b7e5-a296ac56d570 None admin] RBAC: Authorizing `identity:create_grant(project_id=793400b224a54914900adad496869b83, user_id=e5c97ea444ec49788b6a9bd381886011, role_id=dc46bbae6e0f42baaa61e88b45035c46)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:32.267236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940bf205-87c4-4e35-b7e5-a296ac56d570 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:32.278972 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:32.301657 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 143/291] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:54:32.308132 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a35b88-3bda-4328-be98-0fc616939d13 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:32.353024 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 149/292] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 10 16:54:32 2022] PUT /identity/v3/projects/793400b224a54914900adad496869b83/users/e5c97ea444ec49788b6a9bd381886011/roles/dc46bbae6e0f42baaa61e88b45035c46 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:32.440026 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a35b88-3bda-4328-be98-0fc616939d13 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:32.489846 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb39576-4e36-471a-8377-989f24886f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:32.490872 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb39576-4e36-471a-8377-989f24886f87 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:32.491385 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb39576-4e36-471a-8377-989f24886f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:32.491851 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb39576-4e36-471a-8377-989f24886f87 None admin] PATH_INFO: `/v3/projects/793400b224a54914900adad496869b83` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:32.493184 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb39576-4e36-471a-8377-989f24886f87 None admin] RBAC: Authorizing `identity:get_project(project_id=793400b224a54914900adad496869b83)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:32.494866 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb39576-4e36-471a-8377-989f24886f87 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:32.497471 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 144/293] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:32 2022] GET /identity/v3/projects/793400b224a54914900adad496869b83 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:32.503835 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bf205-87c4-4e35-b7e5-a296ac56d570 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:32.653027 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bf205-87c4-4e35-b7e5-a296ac56d570 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:32.689054 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d89ad1-5b43-493b-8363-75ab00480f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:32.689718 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d89ad1-5b43-493b-8363-75ab00480f70 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:32.689994 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d89ad1-5b43-493b-8363-75ab00480f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:32.690268 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d89ad1-5b43-493b-8363-75ab00480f70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:32.691070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d89ad1-5b43-493b-8363-75ab00480f70 None admin] RBAC: Adding query filter params (role.id=dfb72dc1f3744c8e957abcc4ac967dac, scope.project.id=793400b224a54914900adad496869b83, user.id=e9d44b03515c47a7956dc783e3195275) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:32.691380 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d89ad1-5b43-493b-8363-75ab00480f70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:32.692873 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d89ad1-5b43-493b-8363-75ab00480f70 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:32.699039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 150/294] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:54:32 2022] GET /identity/v3/role_assignments?user.id=e9d44b03515c47a7956dc783e3195275&scope.project.id=793400b224a54914900adad496869b83&role.id=dfb72dc1f3744c8e957abcc4ac967dac => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:33.092654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488cea3-18ac-41f3-a782-6848c1b8c523 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:33.093169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488cea3-18ac-41f3-a782-6848c1b8c523 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:33.093419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488cea3-18ac-41f3-a782-6848c1b8c523 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:33.095269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 145/295] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:33.100582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7a632-956d-41f0-ace7-a1634a4b570a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:33.101049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7a632-956d-41f0-ace7-a1634a4b570a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:33.101253 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7a632-956d-41f0-ace7-a1634a4b570a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:33.106216 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bae52f-eff9-4356-9f99-ad1d016851de None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:33.106564 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bae52f-eff9-4356-9f99-ad1d016851de None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:33.106843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bae52f-eff9-4356-9f99-ad1d016851de None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:33.108856 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 146/296] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:33.114323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d968560-80f7-4245-ad5f-28b9df4d8188 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:33.117143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d968560-80f7-4245-ad5f-28b9df4d8188 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:33.117430 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d968560-80f7-4245-ad5f-28b9df4d8188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:33.132554 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5fe7a632-956d-41f0-ace7-a1634a4b570a None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:33.134094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5fe7a632-956d-41f0-ace7-a1634a4b570a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:33.141176 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2d968560-80f7-4245-ad5f-28b9df4d8188 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:33.142435 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2d968560-80f7-4245-ad5f-28b9df4d8188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:33.197840 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:33.203136 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:33.230017 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 151/297] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:33.232287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693918eb-200e-440a-af06-34a1c816f7e6 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:33.238372 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 147/298] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:33.238512 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693918eb-200e-440a-af06-34a1c816f7e6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:33.238512 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693918eb-200e-440a-af06-34a1c816f7e6 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:33.238512 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 152/299] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:33.245076 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfffc08-2a56-4a3a-93d0-0bb4853791d4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:33.245583 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfffc08-2a56-4a3a-93d0-0bb4853791d4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:33.246087 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfffc08-2a56-4a3a-93d0-0bb4853791d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:33.254901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693918eb-200e-440a-af06-34a1c816f7e6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:33.268999 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6cfffc08-2a56-4a3a-93d0-0bb4853791d4 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:33.271740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6cfffc08-2a56-4a3a-93d0-0bb4853791d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:33.347363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 148/300] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:33.348214 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfffc08-2a56-4a3a-93d0-0bb4853791d4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:33.369560 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693918eb-200e-440a-af06-34a1c816f7e6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:33.405875 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62595831-017c-4713-8b05-6bfb282ff909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:33.406588 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62595831-017c-4713-8b05-6bfb282ff909 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:33.406883 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62595831-017c-4713-8b05-6bfb282ff909 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:33.407090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62595831-017c-4713-8b05-6bfb282ff909 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:33.418873 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62595831-017c-4713-8b05-6bfb282ff909 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:33.424128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-62595831-017c-4713-8b05-6bfb282ff909 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:54:33.424128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:33.424128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:33.424128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:33.424128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:33.424128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:33.424128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:33.424128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:33.424128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:33.424128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:33.424128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:33.424128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:33.424128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:33.424128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:33.424128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:33.424128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:33.425049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:33.425049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:33.425049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:33.425049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:33.425049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:33.425049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:33.425049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:33.425049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:33.425049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:33.425049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:33.425049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:33.425049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:33.425049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:33.425049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:33.425049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:33.425049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:33.425838 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:33.425838 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:33.425838 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:33.425838 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:33.425838 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:33.425838 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:33.425838 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:33.425838 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:33.425838 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:33.425838 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:33.425838 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:54:33.425838 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:33.425838 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 153/301] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:33.426510 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62595831-017c-4713-8b05-6bfb282ff909 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:33.463629 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfffc08-2a56-4a3a-93d0-0bb4853791d4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:33.463822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:33.471697 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:33.500260 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292912c-6a7e-4589-a334-62305163f1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:33.501198 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292912c-6a7e-4589-a334-62305163f1bd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:33.501622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292912c-6a7e-4589-a334-62305163f1bd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:33.502037 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292912c-6a7e-4589-a334-62305163f1bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:33.512427 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a292912c-6a7e-4589-a334-62305163f1bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:33.517573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-a292912c-6a7e-4589-a334-62305163f1bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:33.517573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:33.517573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:33.517573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:33.517573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:33.517573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:33.517573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:33.517573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:33.517573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:33.517573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:33.517573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:33.517573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:33.517573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:33.517573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:33.517573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:33.517573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:33.518494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:33.518494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:33.518494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:33.518494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:33.518494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:33.518494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:33.518494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:33.518494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:33.518494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:33.518494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:33.518494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:33.518494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:33.518494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:33.518494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:33.518494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:33.518494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:33.519484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:33.519484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:33.519484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:33.519484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:33.519484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:33.519484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:33.519484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:33.519484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:33.519484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:33.519484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:33.519484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:33.519484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:54:33.524168 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 149/302] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:33.525712 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292912c-6a7e-4589-a334-62305163f1bd None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:33.539293 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62595831-017c-4713-8b05-6bfb282ff909 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:33.576046 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd6a2a-d842-4516-a09e-7d6a2040d7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:33.576669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd6a2a-d842-4516-a09e-7d6a2040d7e7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:33.576936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd6a2a-d842-4516-a09e-7d6a2040d7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:33.577206 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd6a2a-d842-4516-a09e-7d6a2040d7e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:33.578342 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd6a2a-d842-4516-a09e-7d6a2040d7e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:33.579798 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd6a2a-d842-4516-a09e-7d6a2040d7e7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:33.581387 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 154/303] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:54:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:33.582143 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd6a2a-d842-4516-a09e-7d6a2040d7e7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:33.642992 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292912c-6a7e-4589-a334-62305163f1bd None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:33.680460 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b2484-b4ed-4ed2-91fa-26f1b747c5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:33.681063 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b2484-b4ed-4ed2-91fa-26f1b747c5d9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:33.681341 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b2484-b4ed-4ed2-91fa-26f1b747c5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:33.681612 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b2484-b4ed-4ed2-91fa-26f1b747c5d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:33.682272 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b2484-b4ed-4ed2-91fa-26f1b747c5d9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:33.682673 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b2484-b4ed-4ed2-91fa-26f1b747c5d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:33.684194 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b2484-b4ed-4ed2-91fa-26f1b747c5d9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:33.692231 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 150/304] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:33.692994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b2484-b4ed-4ed2-91fa-26f1b747c5d9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:33.700799 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd6a2a-d842-4516-a09e-7d6a2040d7e7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:33.704085 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:33.725291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:33.737466 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685abd70-f5ff-413c-b37a-b7c1e5b665bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:33.738469 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685abd70-f5ff-413c-b37a-b7c1e5b665bb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:33.738469 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685abd70-f5ff-413c-b37a-b7c1e5b665bb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:33.738738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685abd70-f5ff-413c-b37a-b7c1e5b665bb None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:33.739494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685abd70-f5ff-413c-b37a-b7c1e5b665bb None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:33.739786 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685abd70-f5ff-413c-b37a-b7c1e5b665bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:33.741144 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685abd70-f5ff-413c-b37a-b7c1e5b665bb None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:33.749791 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 155/305] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:33.750891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685abd70-f5ff-413c-b37a-b7c1e5b665bb None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:33.800636 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b2484-b4ed-4ed2-91fa-26f1b747c5d9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:33.837040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4be92-184a-4de8-9edb-93155e5f2eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:33.837644 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4be92-184a-4de8-9edb-93155e5f2eb2 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:33.837917 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4be92-184a-4de8-9edb-93155e5f2eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:33.838249 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4be92-184a-4de8-9edb-93155e5f2eb2 None admin] PATH_INFO: `/v3/groups` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:33.838887 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4be92-184a-4de8-9edb-93155e5f2eb2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:33.840918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4be92-184a-4de8-9edb-93155e5f2eb2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:33.867149 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685abd70-f5ff-413c-b37a-b7c1e5b665bb None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:33.899908 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f540123-b5da-4d22-8849-b4005b82f49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:33.900534 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f540123-b5da-4d22-8849-b4005b82f49e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:33.900806 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f540123-b5da-4d22-8849-b4005b82f49e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:33.901073 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f540123-b5da-4d22-8849-b4005b82f49e None admin] PATH_INFO: `/v3/users/0ab1ab6bdec142b98f2114d0769ca116` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:33.922385 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 151/306] 10.222.0.45 () {60 vars in 1134 bytes} [Wed Aug 10 16:54:33 2022] POST /identity/v3/groups => generated 229 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:33.922385 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4be92-184a-4de8-9edb-93155e5f2eb2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:33.923668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f540123-b5da-4d22-8849-b4005b82f49e None admin] RBAC: Authorizing `identity:get_user(user_id=0ab1ab6bdec142b98f2114d0769ca116)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:33.928413 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f540123-b5da-4d22-8849-b4005b82f49e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:33.928413 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 156/307] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:33 2022] GET /identity/v3/users/0ab1ab6bdec142b98f2114d0769ca116 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:33.933383 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f540123-b5da-4d22-8849-b4005b82f49e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:33.966573 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:34.036552 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:34.049380 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4be92-184a-4de8-9edb-93155e5f2eb2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:34.069655 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f540123-b5da-4d22-8849-b4005b82f49e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:34.089588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13198f3b-d7fd-44e8-8e4b-9b02c542aa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:34.090935 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13198f3b-d7fd-44e8-8e4b-9b02c542aa9f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:34.091223 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13198f3b-d7fd-44e8-8e4b-9b02c542aa9f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:34.091506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13198f3b-d7fd-44e8-8e4b-9b02c542aa9f None admin] PATH_INFO: `/v3/users/d166bede7100477b91d4a9ee24b4100f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:34.093489 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13198f3b-d7fd-44e8-8e4b-9b02c542aa9f None admin] RBAC: Authorizing `identity:get_user(user_id=d166bede7100477b91d4a9ee24b4100f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:34.095111 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13198f3b-d7fd-44e8-8e4b-9b02c542aa9f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:34.097016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 152/308] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:33 2022] GET /identity/v3/users/d166bede7100477b91d4a9ee24b4100f => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:34.103032 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13198f3b-d7fd-44e8-8e4b-9b02c542aa9f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:34.111111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18cad5-5ec4-45b8-bf1f-4039348a17c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:34.111757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18cad5-5ec4-45b8-bf1f-4039348a17c1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:34.111978 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18cad5-5ec4-45b8-bf1f-4039348a17c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:34.112294 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18cad5-5ec4-45b8-bf1f-4039348a17c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:34.113018 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b18cad5-5ec4-45b8-bf1f-4039348a17c1 None admin] RBAC: Adding query filter params (role.id=dc46bbae6e0f42baaa61e88b45035c46, user.id=0ab1ab6bdec142b98f2114d0769ca116, scope.system=all) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:34.113317 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b18cad5-5ec4-45b8-bf1f-4039348a17c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:34.114962 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b18cad5-5ec4-45b8-bf1f-4039348a17c1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:34.126529 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 157/309] 10.222.0.45 () {56 vars in 1308 bytes} [Wed Aug 10 16:54:33 2022] GET /identity/v3/role_assignments?user.id=0ab1ab6bdec142b98f2114d0769ca116&scope.system=all&role.id=dc46bbae6e0f42baaa61e88b45035c46 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:34.224250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13198f3b-d7fd-44e8-8e4b-9b02c542aa9f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:34.262874 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d05167-6d45-44ea-9c0c-e002e070bd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:34.264049 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d05167-6d45-44ea-9c0c-e002e070bd4d None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:34.264338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d05167-6d45-44ea-9c0c-e002e070bd4d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:34.264612 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d05167-6d45-44ea-9c0c-e002e070bd4d None admin] PATH_INFO: `/v3/system/users/d166bede7100477b91d4a9ee24b4100f/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:34.266162 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d05167-6d45-44ea-9c0c-e002e070bd4d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d166bede7100477b91d4a9ee24b4100f, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:34.267614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d05167-6d45-44ea-9c0c-e002e070bd4d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:34.388618 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 153/310] 10.222.0.45 () {58 vars in 1334 bytes} [Wed Aug 10 16:54:34 2022] PUT /identity/v3/system/users/d166bede7100477b91d4a9ee24b4100f/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:54:34.708318 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ba9ef-d94f-4e94-a00d-4254e198bce3 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:34.708498 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ba9ef-d94f-4e94-a00d-4254e198bce3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:34.708775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ba9ef-d94f-4e94-a00d-4254e198bce3 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:34.710442 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 158/311] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:34.716141 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42359afc-14b8-4073-9881-fdb352161f18 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:34.716430 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42359afc-14b8-4073-9881-fdb352161f18 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:34.716706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42359afc-14b8-4073-9881-fdb352161f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:34.734738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-42359afc-14b8-4073-9881-fdb352161f18 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:34.735924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-42359afc-14b8-4073-9881-fdb352161f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:34.750656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:34.806596 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 154/312] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:34.829974 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ba9ef-d94f-4e94-a00d-4254e198bce3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:34.923269 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:34.936631 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ba9ef-d94f-4e94-a00d-4254e198bce3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:34.972506 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdc153-c09a-4071-90af-4b15370fe83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:34.973052 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdc153-c09a-4071-90af-4b15370fe83f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:34.973328 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdc153-c09a-4071-90af-4b15370fe83f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:34.973595 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdc153-c09a-4071-90af-4b15370fe83f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:34.983957 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fdc153-c09a-4071-90af-4b15370fe83f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:34.988649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-07fdc153-c09a-4071-90af-4b15370fe83f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:54:34.988649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:34.988649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:34.988649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:34.988649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:34.988649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:34.988649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:34.988649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:34.988649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:34.988649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:34.988649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:34.988649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:34.988649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:34.988649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:34.988649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:34.988649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:34.989516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:34.989516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:34.989516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:34.989516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:34.989516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:34.989516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:34.989516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:34.989516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:34.989516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:34.989516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:34.989516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:34.989516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:34.989516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:34.989516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:34.989516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:34.989516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:34.990325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:34.990325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:34.990325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:34.990325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:34.990325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:34.990325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:34.990325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:34.990325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:34.990325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:34.990325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:34.990325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:54:34.990325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:34.990325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 159/313] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:34.996650 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42359afc-14b8-4073-9881-fdb352161f18 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:35.098508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42359afc-14b8-4073-9881-fdb352161f18 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:35.134456 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc10ba-86c6-4cf1-a7f2-92a646725ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:35.135066 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc10ba-86c6-4cf1-a7f2-92a646725ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:35.135342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc10ba-86c6-4cf1-a7f2-92a646725ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:35.135658 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc10ba-86c6-4cf1-a7f2-92a646725ad8 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:35.136323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc10ba-86c6-4cf1-a7f2-92a646725ad8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:35.136622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc10ba-86c6-4cf1-a7f2-92a646725ad8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:35.138247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc10ba-86c6-4cf1-a7f2-92a646725ad8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:35.147465 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 155/314] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:35.153481 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdc153-c09a-4071-90af-4b15370fe83f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:35.264293 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdc153-c09a-4071-90af-4b15370fe83f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:35.301518 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db500b0c-0f27-4c36-816f-3f7460eb76b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:35.302140 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db500b0c-0f27-4c36-816f-3f7460eb76b4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:35.302449 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db500b0c-0f27-4c36-816f-3f7460eb76b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:35.302721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db500b0c-0f27-4c36-816f-3f7460eb76b4 None admin] PATH_INFO: `/v3/users/e5c97ea444ec49788b6a9bd381886011` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:35.303964 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db500b0c-0f27-4c36-816f-3f7460eb76b4 None admin] RBAC: Authorizing `identity:get_user(user_id=e5c97ea444ec49788b6a9bd381886011)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:35.305418 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db500b0c-0f27-4c36-816f-3f7460eb76b4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:35.307055 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 160/315] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:35 2022] GET /identity/v3/users/e5c97ea444ec49788b6a9bd381886011 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:35.313146 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc10ba-86c6-4cf1-a7f2-92a646725ad8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:35.338802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:35.416903 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc10ba-86c6-4cf1-a7f2-92a646725ad8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:35.452470 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cadc2-250f-43a9-b325-cbbef9de408d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:35.453133 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cadc2-250f-43a9-b325-cbbef9de408d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:35.453416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cadc2-250f-43a9-b325-cbbef9de408d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:35.453686 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cadc2-250f-43a9-b325-cbbef9de408d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:35.523303 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1cadc2-250f-43a9-b325-cbbef9de408d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:35.525003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1cadc2-250f-43a9-b325-cbbef9de408d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:35.546699 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:35.636845 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 156/316] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:54:35.642555 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db500b0c-0f27-4c36-816f-3f7460eb76b4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:35.667523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:35.750205 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db500b0c-0f27-4c36-816f-3f7460eb76b4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:35.788711 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb101fda-1da4-4352-abfc-60042922813d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:35.789666 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb101fda-1da4-4352-abfc-60042922813d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:35.790145 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb101fda-1da4-4352-abfc-60042922813d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:35.790656 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb101fda-1da4-4352-abfc-60042922813d None admin] PATH_INFO: `/v3/projects/793400b224a54914900adad496869b83` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:35.791982 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb101fda-1da4-4352-abfc-60042922813d None admin] RBAC: Authorizing `identity:get_project(project_id=793400b224a54914900adad496869b83)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:35.793713 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb101fda-1da4-4352-abfc-60042922813d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:35.795608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 161/317] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:35 2022] GET /identity/v3/projects/793400b224a54914900adad496869b83 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:35.801727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cadc2-250f-43a9-b325-cbbef9de408d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:35.906488 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cadc2-250f-43a9-b325-cbbef9de408d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:35.920539 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:35.942240 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7ad3b-9e27-4a3a-a7f4-699c4a027d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:35.942883 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7ad3b-9e27-4a3a-a7f4-699c4a027d5a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:35.945093 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7ad3b-9e27-4a3a-a7f4-699c4a027d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:35.945093 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7ad3b-9e27-4a3a-a7f4-699c4a027d5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:35.945093 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f7ad3b-9e27-4a3a-a7f4-699c4a027d5a None admin] RBAC: Adding query filter params (role.id=dc46bbae6e0f42baaa61e88b45035c46, scope.project.id=793400b224a54914900adad496869b83, user.id=e5c97ea444ec49788b6a9bd381886011) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:35.945093 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f7ad3b-9e27-4a3a-a7f4-699c4a027d5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:35.945896 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f7ad3b-9e27-4a3a-a7f4-699c4a027d5a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:35.953040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 157/318] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:54:35 2022] GET /identity/v3/role_assignments?user.id=e5c97ea444ec49788b6a9bd381886011&scope.project.id=793400b224a54914900adad496869b83&role.id=dc46bbae6e0f42baaa61e88b45035c46 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:36.109183 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166e9c9-8b60-4670-913b-c031335a528d None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:36.109466 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166e9c9-8b60-4670-913b-c031335a528d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:36.109733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166e9c9-8b60-4670-913b-c031335a528d None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:36.111375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 162/319] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:36.116560 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5c6ff-1523-4811-86b9-d20e5005cbc7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:36.117394 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5c6ff-1523-4811-86b9-d20e5005cbc7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:36.117906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5c6ff-1523-4811-86b9-d20e5005cbc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:36.138689 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c8b5c6ff-1523-4811-86b9-d20e5005cbc7 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:36.140232 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c8b5c6ff-1523-4811-86b9-d20e5005cbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:36.203429 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a88a1-a223-429a-a93f-8b7a0da9a758 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:36.203934 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a88a1-a223-429a-a93f-8b7a0da9a758 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:36.203934 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a88a1-a223-429a-a93f-8b7a0da9a758 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:36.205281 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 163/320] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:54:36.211568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b96900-b716-4b1a-aa74-6293274ce62e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:36.211960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b96900-b716-4b1a-aa74-6293274ce62e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:36.212229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b96900-b716-4b1a-aa74-6293274ce62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:36.216669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 158/321] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:36.231651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-09b96900-b716-4b1a-aa74-6293274ce62e None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:36.232847 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-09b96900-b716-4b1a-aa74-6293274ce62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:36.241056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5c6ff-1523-4811-86b9-d20e5005cbc7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:36.307476 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 164/322] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:54:36.329746 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b96900-b716-4b1a-aa74-6293274ce62e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:36.345649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:36.348662 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5c6ff-1523-4811-86b9-d20e5005cbc7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:36.357299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:36.384004 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28871ca3-1d33-4818-9031-3e45b4e29973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:36.384558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28871ca3-1d33-4818-9031-3e45b4e29973 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:36.384831 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28871ca3-1d33-4818-9031-3e45b4e29973 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:36.385105 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28871ca3-1d33-4818-9031-3e45b4e29973 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:36.386193 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28871ca3-1d33-4818-9031-3e45b4e29973 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:36.387751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28871ca3-1d33-4818-9031-3e45b4e29973 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:36.394670 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 159/323] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:54:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:36.395900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28871ca3-1d33-4818-9031-3e45b4e29973 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:36.437560 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b96900-b716-4b1a-aa74-6293274ce62e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:36.473333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb60b91-bb92-484f-b3ee-652fc19a9f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:36.473959 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb60b91-bb92-484f-b3ee-652fc19a9f8d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:36.474373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb60b91-bb92-484f-b3ee-652fc19a9f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:36.474654 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb60b91-bb92-484f-b3ee-652fc19a9f8d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:36.486312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb60b91-bb92-484f-b3ee-652fc19a9f8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:36.491267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-efb60b91-bb92-484f-b3ee-652fc19a9f8d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:54:36.491267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:36.491267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:36.491267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:36.491267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:36.491267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:36.491267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:36.491267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:36.491267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:36.491267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:36.491267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:36.491267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:36.491267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:36.491267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:36.491267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:36.491267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:36.492134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:36.492134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:36.492134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:36.492134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:36.492134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:36.492134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:36.492134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:36.492134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:36.492134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:36.492134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:36.492134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:36.492134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:36.492134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:36.492134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:36.492134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:36.492134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:36.492980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:36.492980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:36.492980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:36.492980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:36.492980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:36.492980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:36.492980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:36.492980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:36.492980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:36.492980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:36.492980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:54:36.492980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:36.492980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 165/324] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:54:36.498460 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb60b91-bb92-484f-b3ee-652fc19a9f8d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:36.503133 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28871ca3-1d33-4818-9031-3e45b4e29973 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:36.537058 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5967fed-9280-4b18-9673-9f1fb9670068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:36.537658 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5967fed-9280-4b18-9673-9f1fb9670068 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:36.537936 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5967fed-9280-4b18-9673-9f1fb9670068 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:36.538204 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5967fed-9280-4b18-9673-9f1fb9670068 None admin] PATH_INFO: `/v3/groups` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:36.538848 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5967fed-9280-4b18-9673-9f1fb9670068 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:36.540277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5967fed-9280-4b18-9673-9f1fb9670068 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:36.574577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:36.599164 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb60b91-bb92-484f-b3ee-652fc19a9f8d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:36.632758 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc52fed-bbbc-4bf5-903f-5f4a6e14a969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:36.633346 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc52fed-bbbc-4bf5-903f-5f4a6e14a969 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:36.633620 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc52fed-bbbc-4bf5-903f-5f4a6e14a969 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:36.633892 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc52fed-bbbc-4bf5-903f-5f4a6e14a969 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:36.634628 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc52fed-bbbc-4bf5-903f-5f4a6e14a969 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:36.634881 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc52fed-bbbc-4bf5-903f-5f4a6e14a969 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:36.636261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc52fed-bbbc-4bf5-903f-5f4a6e14a969 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:36.644116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 166/325] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:36.645381 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad47b9-7e99-4f3d-b605-dc242ef8d517 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:36.645662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad47b9-7e99-4f3d-b605-dc242ef8d517 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:36.645930 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad47b9-7e99-4f3d-b605-dc242ef8d517 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:36.647336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 167/326] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:36.649442 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad47b9-7e99-4f3d-b605-dc242ef8d517 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:36.658284 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 160/327] 10.222.0.45 () {60 vars in 1134 bytes} [Wed Aug 10 16:54:36 2022] POST /identity/v3/groups => generated 226 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:36.659545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3fefe-b1b0-4d75-b79c-1f2f08badf83 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:36.666168 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3fefe-b1b0-4d75-b79c-1f2f08badf83 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:36.666168 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3fefe-b1b0-4d75-b79c-1f2f08badf83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:36.679961 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-81f3fefe-b1b0-4d75-b79c-1f2f08badf83 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:36.681177 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-81f3fefe-b1b0-4d75-b79c-1f2f08badf83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:36.711239 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:36.757802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 161/328] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:36.771920 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad47b9-7e99-4f3d-b605-dc242ef8d517 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:36.781427 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3fefe-b1b0-4d75-b79c-1f2f08badf83 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:36.814189 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87481d2-db8a-4651-ab98-c384abbe544d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:36.815660 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87481d2-db8a-4651-ab98-c384abbe544d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:36.816584 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87481d2-db8a-4651-ab98-c384abbe544d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:36.816584 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87481d2-db8a-4651-ab98-c384abbe544d None admin] PATH_INFO: `/v3/users/0ab1ab6bdec142b98f2114d0769ca116` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:36.817325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87481d2-db8a-4651-ab98-c384abbe544d None admin] RBAC: Authorizing `identity:get_user(user_id=0ab1ab6bdec142b98f2114d0769ca116)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:36.818793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87481d2-db8a-4651-ab98-c384abbe544d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:36.820378 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 168/329] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:36 2022] GET /identity/v3/users/0ab1ab6bdec142b98f2114d0769ca116 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:36.826088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87481d2-db8a-4651-ab98-c384abbe544d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:36.829746 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:36.899860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3fefe-b1b0-4d75-b79c-1f2f08badf83 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:36.940686 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25056447-04d1-4be0-8d4f-7edc14285bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:36.941669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25056447-04d1-4be0-8d4f-7edc14285bbe None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:36.942213 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25056447-04d1-4be0-8d4f-7edc14285bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:36.942520 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25056447-04d1-4be0-8d4f-7edc14285bbe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:36.945283 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87481d2-db8a-4651-ab98-c384abbe544d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:36.956382 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25056447-04d1-4be0-8d4f-7edc14285bbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:36.961154 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-25056447-04d1-4be0-8d4f-7edc14285bbe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:36.961154 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:36.961154 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:36.961154 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:36.961154 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:36.961154 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:36.961154 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:36.961154 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:36.961154 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:36.961154 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:36.961154 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:36.961154 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:36.961154 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:36.961154 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:36.961154 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:36.961154 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:36.962072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:36.962072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:36.962072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:36.962072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:36.962072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:36.962072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:36.962072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:36.962072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:36.962072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:36.962072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:36.962072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:36.962072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:36.962072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:36.962072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:36.962072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:36.962072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:36.962893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:36.962893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:36.962893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:36.962893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:36.962893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:36.962893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:36.962893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:36.962893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:36.962893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:36.962893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:36.962893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:36.962893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:54:36.966452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 162/330] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:54:36.968670 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25056447-04d1-4be0-8d4f-7edc14285bbe None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:36.986890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd473f5c-7a60-4a26-8697-9ca6614ab760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:36.987544 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd473f5c-7a60-4a26-8697-9ca6614ab760 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:36.987821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd473f5c-7a60-4a26-8697-9ca6614ab760 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:36.988096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd473f5c-7a60-4a26-8697-9ca6614ab760 None admin] PATH_INFO: `/v3/system/users/0ab1ab6bdec142b98f2114d0769ca116/roles/dc46bbae6e0f42baaa61e88b45035c46` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:36.989628 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd473f5c-7a60-4a26-8697-9ca6614ab760 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0ab1ab6bdec142b98f2114d0769ca116, role_id=dc46bbae6e0f42baaa61e88b45035c46)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:36.991516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd473f5c-7a60-4a26-8697-9ca6614ab760 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:37.000327 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:37.072626 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 169/331] 10.222.0.45 () {58 vars in 1334 bytes} [Wed Aug 10 16:54:36 2022] PUT /identity/v3/system/users/0ab1ab6bdec142b98f2114d0769ca116/roles/dc46bbae6e0f42baaa61e88b45035c46 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:37.082616 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25056447-04d1-4be0-8d4f-7edc14285bbe None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:37.129263 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8f54b-7fde-40bb-a1be-6ec43baae98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:37.129913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8f54b-7fde-40bb-a1be-6ec43baae98b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:37.130168 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8f54b-7fde-40bb-a1be-6ec43baae98b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:37.130439 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8f54b-7fde-40bb-a1be-6ec43baae98b None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:37.136354 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8f54b-7fde-40bb-a1be-6ec43baae98b None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:37.136354 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8f54b-7fde-40bb-a1be-6ec43baae98b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:37.136354 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8f54b-7fde-40bb-a1be-6ec43baae98b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:37.143003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 163/332] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:37.149122 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd473f5c-7a60-4a26-8697-9ca6614ab760 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:37.234094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:37.274254 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd473f5c-7a60-4a26-8697-9ca6614ab760 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:37.312266 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da31d2-07ef-4838-89b3-a606b20ca301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:37.313017 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da31d2-07ef-4838-89b3-a606b20ca301 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:37.313344 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da31d2-07ef-4838-89b3-a606b20ca301 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:37.313618 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da31d2-07ef-4838-89b3-a606b20ca301 None admin] PATH_INFO: `/v3/users/d166bede7100477b91d4a9ee24b4100f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:37.314807 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da31d2-07ef-4838-89b3-a606b20ca301 None admin] RBAC: Authorizing `identity:get_user(user_id=d166bede7100477b91d4a9ee24b4100f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:37.316261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da31d2-07ef-4838-89b3-a606b20ca301 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:37.317864 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 170/333] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:37 2022] GET /identity/v3/users/d166bede7100477b91d4a9ee24b4100f => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:37.323786 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8f54b-7fde-40bb-a1be-6ec43baae98b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:37.417533 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:37.440028 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8f54b-7fde-40bb-a1be-6ec43baae98b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:37.479016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbd3c8-f8e8-4953-ba99-a031a896cdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:37.479626 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbd3c8-f8e8-4953-ba99-a031a896cdec None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:37.479904 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbd3c8-f8e8-4953-ba99-a031a896cdec None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:37.480316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbd3c8-f8e8-4953-ba99-a031a896cdec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:37.481612 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfbd3c8-f8e8-4953-ba99-a031a896cdec None admin] RBAC: Adding query filter params (role.id=dfb72dc1f3744c8e957abcc4ac967dac, user.id=d166bede7100477b91d4a9ee24b4100f, scope.system=all) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:37.481977 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfbd3c8-f8e8-4953-ba99-a031a896cdec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:37.483461 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfbd3c8-f8e8-4953-ba99-a031a896cdec None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:37.499454 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 164/334] 10.222.0.45 () {56 vars in 1308 bytes} [Wed Aug 10 16:54:37 2022] GET /identity/v3/role_assignments?user.id=d166bede7100477b91d4a9ee24b4100f&scope.system=all&role.id=dfb72dc1f3744c8e957abcc4ac967dac => generated 510 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:38.978642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdb383-46b0-4311-ad69-cb8fbcb6fb24 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:38.979116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdb383-46b0-4311-ad69-cb8fbcb6fb24 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:38.979116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdb383-46b0-4311-ad69-cb8fbcb6fb24 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:38.980775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 171/335] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:38.985938 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539908b-3dc4-4caa-83c5-0e89c4125482 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:38.986334 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539908b-3dc4-4caa-83c5-0e89c4125482 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:38.986572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539908b-3dc4-4caa-83c5-0e89c4125482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:39.013072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c539908b-3dc4-4caa-83c5-0e89c4125482 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:39.014343 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c539908b-3dc4-4caa-83c5-0e89c4125482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:39.098342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 165/336] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:39.140545 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebdb383-46b0-4311-ad69-cb8fbcb6fb24 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:39.264712 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebdb383-46b0-4311-ad69-cb8fbcb6fb24 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:39.301344 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf72f0-f06b-4cb4-902e-5101a3e7211d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:39.301658 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf72f0-f06b-4cb4-902e-5101a3e7211d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:39.301897 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf72f0-f06b-4cb4-902e-5101a3e7211d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:39.303551 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 166/337] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:54:39.307848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4755acdc-1628-4162-960e-35314716b697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:39.309150 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755acdc-1628-4162-960e-35314716b697 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:39.309497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755acdc-1628-4162-960e-35314716b697 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:39.309766 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755acdc-1628-4162-960e-35314716b697 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:39.310758 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722fad3f-536e-48f7-8aa3-6c7b2f5ca822 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:39.311043 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722fad3f-536e-48f7-8aa3-6c7b2f5ca822 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:39.311321 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722fad3f-536e-48f7-8aa3-6c7b2f5ca822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:39.313718 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:39.326498 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4755acdc-1628-4162-960e-35314716b697 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:39.331390 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-4755acdc-1628-4162-960e-35314716b697 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:39.331390 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:39.331390 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:39.331390 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:39.331390 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:39.331390 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:39.331390 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:39.331390 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:39.331390 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:39.331390 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:39.331390 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:39.331390 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:39.331390 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:39.331390 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:39.331390 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:39.331390 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:39.335152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:39.335152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:39.335152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:39.335152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:39.335152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:39.335152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:39.335152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:39.335152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:39.335152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:39.335152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:39.335152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:39.335152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:39.335152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:39.335152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:39.335152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:39.335152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:39.335956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:39.335956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:39.335956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:39.335956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:39.335956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:39.335956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:39.335956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:39.335956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:39.335956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:39.335956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:39.335956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:39.335956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:39.335956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 172/338] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:39.348790 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-722fad3f-536e-48f7-8aa3-6c7b2f5ca822 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:39.349085 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4755acdc-1628-4162-960e-35314716b697 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:39.350103 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-722fad3f-536e-48f7-8aa3-6c7b2f5ca822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:39.533637 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 167/339] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:39.547875 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4755acdc-1628-4162-960e-35314716b697 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:39.569638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5dad3-a31d-45a9-a557-c16027b942a3 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:39.569638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5dad3-a31d-45a9-a557-c16027b942a3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:39.570030 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5dad3-a31d-45a9-a557-c16027b942a3 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:39.571639 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 168/340] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:54:39.578042 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e31b80-ed89-4aec-8d4c-2e2529b865c0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:39.578326 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e31b80-ed89-4aec-8d4c-2e2529b865c0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:39.578630 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e31b80-ed89-4aec-8d4c-2e2529b865c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:39.594513 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:39.608233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77b2ef-5a66-41cb-9c47-3e76197808af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:39.619913 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77b2ef-5a66-41cb-9c47-3e76197808af None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:39.619913 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77b2ef-5a66-41cb-9c47-3e76197808af None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:39.619913 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77b2ef-5a66-41cb-9c47-3e76197808af None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:39.619913 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77b2ef-5a66-41cb-9c47-3e76197808af None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:39.619913 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77b2ef-5a66-41cb-9c47-3e76197808af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:39.620761 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-57e31b80-ed89-4aec-8d4c-2e2529b865c0 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:39.620839 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77b2ef-5a66-41cb-9c47-3e76197808af None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:39.621438 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-57e31b80-ed89-4aec-8d4c-2e2529b865c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:39.627248 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 173/341] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:39.627394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77b2ef-5a66-41cb-9c47-3e76197808af None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:39.722436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 169/342] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:54:39.723311 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e31b80-ed89-4aec-8d4c-2e2529b865c0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:39.765662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:39.841910 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77b2ef-5a66-41cb-9c47-3e76197808af None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:39.896372 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e31b80-ed89-4aec-8d4c-2e2529b865c0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:39.909546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffcff33-0d9a-445b-a044-f6ae9087d439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:39.910182 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcff33-0d9a-445b-a044-f6ae9087d439 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:39.910495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcff33-0d9a-445b-a044-f6ae9087d439 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:39.910744 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcff33-0d9a-445b-a044-f6ae9087d439 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:39.935229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffcff33-0d9a-445b-a044-f6ae9087d439 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:39.943198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-6ffcff33-0d9a-445b-a044-f6ae9087d439 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:54:39.943198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:39.943198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:39.943198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:39.943198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:39.943198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:39.943198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:39.943198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:39.943198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:39.943198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:39.943198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:39.943198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:39.943198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:39.943198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:39.943198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:39.943198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:39.944092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:39.944092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:39.944092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:39.944092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:39.944092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:39.944092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:39.944092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:39.944092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:39.944092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:39.944092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:39.944092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:39.944092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:39.944092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:39.944092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:39.944092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:39.944092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:39.944909 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:39.944909 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:39.944909 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:39.944909 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:39.944909 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:39.944909 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:39.944909 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:39.944909 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:39.944909 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:39.944909 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:39.944909 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:54:39.944909 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:39.944909 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 174/343] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 10 16:54:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:39.945568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffcff33-0d9a-445b-a044-f6ae9087d439 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:39.966500 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff855d0f-7785-4b5a-a29d-e6d4f6059a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:39.967113 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff855d0f-7785-4b5a-a29d-e6d4f6059a97 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:39.967388 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff855d0f-7785-4b5a-a29d-e6d4f6059a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:39.967654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff855d0f-7785-4b5a-a29d-e6d4f6059a97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:39.972298 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:40.073886 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff855d0f-7785-4b5a-a29d-e6d4f6059a97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:40.076045 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff855d0f-7785-4b5a-a29d-e6d4f6059a97 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:40.107801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffcff33-0d9a-445b-a044-f6ae9087d439 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:40.129096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:40.170299 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57494994-4196-4abc-a742-d4fbd50e81b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:40.171341 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57494994-4196-4abc-a742-d4fbd50e81b4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.171987 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57494994-4196-4abc-a742-d4fbd50e81b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.173466 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57494994-4196-4abc-a742-d4fbd50e81b4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.195061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57494994-4196-4abc-a742-d4fbd50e81b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:40.201456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-57494994-4196-4abc-a742-d4fbd50e81b4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:54:40.201456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:40.201456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:40.201456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:40.201456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:40.201456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:40.201456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:40.201456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:40.201456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:40.201456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:40.201456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:40.201456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:40.201456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:40.201456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:40.201456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:40.201456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:40.202408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:40.202408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:40.202408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:40.202408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:40.202408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:40.202408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:40.202408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:40.202408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:40.202408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:40.202408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:40.202408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:40.202408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:40.202408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:40.202408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:40.202408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:40.202408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:40.203325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:40.203325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:40.203325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:40.203325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:40.203325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:40.203325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:40.203325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:40.203325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:40.203325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:40.203325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:40.203325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 16:54:40.203325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:40.210400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 175/344] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:40.218013 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57494994-4196-4abc-a742-d4fbd50e81b4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.256051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 170/345] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:54:40.257272 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff855d0f-7785-4b5a-a29d-e6d4f6059a97 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.300055 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:40.347007 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57494994-4196-4abc-a742-d4fbd50e81b4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:40.375148 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff855d0f-7785-4b5a-a29d-e6d4f6059a97 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:40.381919 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e048b95-8fee-46e4-be98-1afd58374a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:40.382549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e048b95-8fee-46e4-be98-1afd58374a29 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.382825 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e048b95-8fee-46e4-be98-1afd58374a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.383094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e048b95-8fee-46e4-be98-1afd58374a29 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.383771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e048b95-8fee-46e4-be98-1afd58374a29 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:40.384070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e048b95-8fee-46e4-be98-1afd58374a29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:40.385459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e048b95-8fee-46e4-be98-1afd58374a29 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:40.394029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 176/346] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:54:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:40.394967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e048b95-8fee-46e4-be98-1afd58374a29 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.416266 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed487aa-e423-485d-8f1c-32605eb1c369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:40.417252 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed487aa-e423-485d-8f1c-32605eb1c369 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.417731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed487aa-e423-485d-8f1c-32605eb1c369 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.418808 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed487aa-e423-485d-8f1c-32605eb1c369 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.419960 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed487aa-e423-485d-8f1c-32605eb1c369 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:40.420462 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed487aa-e423-485d-8f1c-32605eb1c369 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:40.422069 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed487aa-e423-485d-8f1c-32605eb1c369 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:40.431286 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 171/347] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:40.432744 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af55241-510a-44d4-87ed-c0460305ecb1 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.433237 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af55241-510a-44d4-87ed-c0460305ecb1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.433771 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af55241-510a-44d4-87ed-c0460305ecb1 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.435791 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 172/348] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:40.436803 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af55241-510a-44d4-87ed-c0460305ecb1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.450787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:40.504164 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e048b95-8fee-46e4-be98-1afd58374a29 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:40.557898 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33713644-3820-4b0a-96a1-eb9daf7cb401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:40.558640 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33713644-3820-4b0a-96a1-eb9daf7cb401 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.558920 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33713644-3820-4b0a-96a1-eb9daf7cb401 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.559188 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33713644-3820-4b0a-96a1-eb9daf7cb401 None admin] PATH_INFO: `/v3/projects/1570ef44f3134332b1fc668345a37479` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.562083 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33713644-3820-4b0a-96a1-eb9daf7cb401 None admin] RBAC: Authorizing `identity:get_project(project_id=1570ef44f3134332b1fc668345a37479)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:40.563862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33713644-3820-4b0a-96a1-eb9daf7cb401 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:40.565334 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:40.565683 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 177/349] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:40 2022] GET /identity/v3/projects/1570ef44f3134332b1fc668345a37479 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:40.570382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e705ff-da1d-4f2d-8694-da2bb9aa4b08 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.570382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e705ff-da1d-4f2d-8694-da2bb9aa4b08 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.570382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e705ff-da1d-4f2d-8694-da2bb9aa4b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.581090 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af55241-510a-44d4-87ed-c0460305ecb1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:40.593841 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b2e705ff-da1d-4f2d-8694-da2bb9aa4b08 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:40.594602 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b2e705ff-da1d-4f2d-8694-da2bb9aa4b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:40.638937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f11c0-46ac-442e-a645-c2c6cdc49c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:40.639134 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f11c0-46ac-442e-a645-c2c6cdc49c42 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.639631 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f11c0-46ac-442e-a645-c2c6cdc49c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.640002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f11c0-46ac-442e-a645-c2c6cdc49c42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.704067 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 178/350] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:40.705082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e705ff-da1d-4f2d-8694-da2bb9aa4b08 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.743530 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14f11c0-46ac-442e-a645-c2c6cdc49c42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:40.746757 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14f11c0-46ac-442e-a645-c2c6cdc49c42 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:40.778743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:40.825646 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e705ff-da1d-4f2d-8694-da2bb9aa4b08 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:40.870106 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:40.872250 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec128554-8241-4460-9121-6c843f34ddfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:40.873278 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec128554-8241-4460-9121-6c843f34ddfe None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:40.874099 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec128554-8241-4460-9121-6c843f34ddfe None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:40.874849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec128554-8241-4460-9121-6c843f34ddfe None admin] PATH_INFO: `/v3/users/0ab1ab6bdec142b98f2114d0769ca116` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:40.876224 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec128554-8241-4460-9121-6c843f34ddfe None admin] RBAC: Authorizing `identity:get_user(user_id=0ab1ab6bdec142b98f2114d0769ca116)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:40.877255 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 173/351] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:54:40.878068 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f11c0-46ac-442e-a645-c2c6cdc49c42 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:40.878741 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec128554-8241-4460-9121-6c843f34ddfe None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:40.887415 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 179/352] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:54:40 2022] GET /identity/v3/users/0ab1ab6bdec142b98f2114d0769ca116 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:40.887415 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec128554-8241-4460-9121-6c843f34ddfe None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.008439 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f11c0-46ac-442e-a645-c2c6cdc49c42 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.016156 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec128554-8241-4460-9121-6c843f34ddfe None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.025349 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:41.052732 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28de644-dd5b-4a6a-a32d-d9af25b84edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.053369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28de644-dd5b-4a6a-a32d-d9af25b84edf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.053642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28de644-dd5b-4a6a-a32d-d9af25b84edf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.053915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28de644-dd5b-4a6a-a32d-d9af25b84edf None admin] PATH_INFO: `/v3/groups/0271954fa90a4c699c1864243d5b4040` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.063637 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332a0b4-89c6-4f49-9783-9584900832dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.067628 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332a0b4-89c6-4f49-9783-9584900832dd None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.067628 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332a0b4-89c6-4f49-9783-9584900832dd None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.067628 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332a0b4-89c6-4f49-9783-9584900832dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.067837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28de644-dd5b-4a6a-a32d-d9af25b84edf None admin] RBAC: Authorizing `identity:get_group(group_id=0271954fa90a4c699c1864243d5b4040)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.068082 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28de644-dd5b-4a6a-a32d-d9af25b84edf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.069925 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 174/353] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 10 16:54:40 2022] GET /identity/v3/groups/0271954fa90a4c699c1864243d5b4040 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:41.071862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28de644-dd5b-4a6a-a32d-d9af25b84edf None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.085516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d332a0b4-89c6-4f49-9783-9584900832dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:41.091406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d332a0b4-89c6-4f49-9783-9584900832dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:41.091406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:41.091406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:41.091406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:41.091406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:41.091406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:41.091406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:41.091406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:41.091406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:41.091406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:41.091406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:41.091406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:41.091406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:41.091406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:41.091406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:41.091406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:41.097312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:41.097312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:41.097312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:41.097312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:41.097312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:41.097312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:41.097312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:41.097312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:41.097312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:41.097312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:41.097312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:41.097312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:41.097312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:41.097312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:41.097312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:41.097312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:41.098157 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:41.098157 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:41.098157 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:41.098157 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:41.098157 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:41.098157 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:41.098157 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:41.098157 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:41.098157 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:41.098157 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:41.098157 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:41.098157 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:41.098157 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 180/354] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:41.098157 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332a0b4-89c6-4f49-9783-9584900832dd None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.151207 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:41.202018 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28de644-dd5b-4a6a-a32d-d9af25b84edf None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.239899 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332a0b4-89c6-4f49-9783-9584900832dd None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.250569 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb8796-c241-4c4f-a18f-c70f85300805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.251078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb8796-c241-4c4f-a18f-c70f85300805 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.251353 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb8796-c241-4c4f-a18f-c70f85300805 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.251625 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb8796-c241-4c4f-a18f-c70f85300805 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.252351 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb8796-c241-4c4f-a18f-c70f85300805 None admin] RBAC: Adding query filter params (role.id=dc46bbae6e0f42baaa61e88b45035c46, user.id=0ab1ab6bdec142b98f2114d0769ca116, scope.system=all) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:41.252966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb8796-c241-4c4f-a18f-c70f85300805 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.254107 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb8796-c241-4c4f-a18f-c70f85300805 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.271093 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 175/355] 10.222.0.45 () {56 vars in 1308 bytes} [Wed Aug 10 16:54:41 2022] GET /identity/v3/role_assignments?user.id=0ab1ab6bdec142b98f2114d0769ca116&scope.system=all&role.id=dc46bbae6e0f42baaa61e88b45035c46 => generated 510 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:41.272277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dee88b-efe4-4108-a32e-488e14fe01d4 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.272560 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dee88b-efe4-4108-a32e-488e14fe01d4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.272831 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dee88b-efe4-4108-a32e-488e14fe01d4 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.274299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 176/356] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:41.283332 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dee88b-efe4-4108-a32e-488e14fe01d4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.288600 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ce148-d3a1-4594-bae6-9524564a2dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.289577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ce148-d3a1-4594-bae6-9524564a2dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.290059 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ce148-d3a1-4594-bae6-9524564a2dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.290602 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ce148-d3a1-4594-bae6-9524564a2dd2 None admin] PATH_INFO: `/v3/projects/1570ef44f3134332b1fc668345a37479` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.292230 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ce148-d3a1-4594-bae6-9524564a2dd2 None admin] RBAC: Authorizing `identity:get_project(project_id=1570ef44f3134332b1fc668345a37479)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.296262 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ce148-d3a1-4594-bae6-9524564a2dd2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.299326 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 181/357] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:41 2022] GET /identity/v3/projects/1570ef44f3134332b1fc668345a37479 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:41.300814 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ce148-d3a1-4594-bae6-9524564a2dd2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.354536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:41.416919 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dee88b-efe4-4108-a32e-488e14fe01d4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.437002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:41.439571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ce148-d3a1-4594-bae6-9524564a2dd2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.458868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0388bdf-c382-4103-9939-68bb70820d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.459890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0388bdf-c382-4103-9939-68bb70820d3e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.460362 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0388bdf-c382-4103-9939-68bb70820d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.460822 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0388bdf-c382-4103-9939-68bb70820d3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.461750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0388bdf-c382-4103-9939-68bb70820d3e None admin] RBAC: Adding query filter params (group.id=0271954fa90a4c699c1864243d5b4040, role.id=dfb72dc1f3744c8e957abcc4ac967dac, scope.project.id=1570ef44f3134332b1fc668345a37479) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:41.462268 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0388bdf-c382-4103-9939-68bb70820d3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.463954 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0388bdf-c382-4103-9939-68bb70820d3e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.470757 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 177/358] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 10 16:54:41 2022] GET /identity/v3/role_assignments?group.id=0271954fa90a4c699c1864243d5b4040&scope.project.id=1570ef44f3134332b1fc668345a37479&role.id=dfb72dc1f3744c8e957abcc4ac967dac => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:41.472257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a05ddd-9db1-4f05-91b2-871ac9c051f6 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.472735 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a05ddd-9db1-4f05-91b2-871ac9c051f6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.473201 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a05ddd-9db1-4f05-91b2-871ac9c051f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.490111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490cf57-8766-42d3-abdb-3cf4870748f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.491186 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490cf57-8766-42d3-abdb-3cf4870748f5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.491683 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490cf57-8766-42d3-abdb-3cf4870748f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.492147 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490cf57-8766-42d3-abdb-3cf4870748f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.493754 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2490cf57-8766-42d3-abdb-3cf4870748f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:41.494288 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2490cf57-8766-42d3-abdb-3cf4870748f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.496901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2490cf57-8766-42d3-abdb-3cf4870748f5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.500425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-85a05ddd-9db1-4f05-91b2-871ac9c051f6 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:41.501731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-85a05ddd-9db1-4f05-91b2-871ac9c051f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:41.510310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 182/359] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:41.512181 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490cf57-8766-42d3-abdb-3cf4870748f5 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.617324 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 178/360] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:41.617324 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a05ddd-9db1-4f05-91b2-871ac9c051f6 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.677950 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:41.687103 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490cf57-8766-42d3-abdb-3cf4870748f5 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.733841 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b1069-f14c-45ea-ae99-31de38540a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.734514 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b1069-f14c-45ea-ae99-31de38540a86 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.734902 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b1069-f14c-45ea-ae99-31de38540a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.735175 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b1069-f14c-45ea-ae99-31de38540a86 None admin] PATH_INFO: `/v3/groups/0271954fa90a4c699c1864243d5b4040` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.736302 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b1069-f14c-45ea-ae99-31de38540a86 None admin] RBAC: Authorizing `identity:get_group(group_id=0271954fa90a4c699c1864243d5b4040)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.737745 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b1069-f14c-45ea-ae99-31de38540a86 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.739526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 183/361] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 10 16:54:41 2022] GET /identity/v3/groups/0271954fa90a4c699c1864243d5b4040 => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:41.740115 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b1069-f14c-45ea-ae99-31de38540a86 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:41.771922 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a05ddd-9db1-4f05-91b2-871ac9c051f6 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.788859 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:41.816368 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b1ad4-a00b-4d99-b201-7396d984bcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.816983 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b1ad4-a00b-4d99-b201-7396d984bcdc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.817260 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b1ad4-a00b-4d99-b201-7396d984bcdc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.817528 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b1ad4-a00b-4d99-b201-7396d984bcdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.873918 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b1069-f14c-45ea-ae99-31de38540a86 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:41.909626 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b1ad4-a00b-4d99-b201-7396d984bcdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:41.909626 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b1ad4-a00b-4d99-b201-7396d984bcdc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:41.927648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952b654-ff77-4bb9-8876-9449c740ff7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:41.928825 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952b654-ff77-4bb9-8876-9449c740ff7e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:41.929307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952b654-ff77-4bb9-8876-9449c740ff7e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:41.929749 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952b654-ff77-4bb9-8876-9449c740ff7e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:41.947897 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a952b654-ff77-4bb9-8876-9449c740ff7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:41.954854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-a952b654-ff77-4bb9-8876-9449c740ff7e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:54:41.954854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:41.954854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:41.954854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:41.954854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:41.954854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:41.954854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:41.954854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:41.954854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:41.954854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:41.954854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:41.954854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:41.954854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:41.954854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:41.954854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:41.954854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:41.955854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:41.955854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:41.955854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:41.955854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:41.955854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:41.955854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:41.955854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:41.955854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:41.955854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:41.955854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:41.955854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:41.955854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:41.955854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:41.955854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:41.955854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:41.955854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:41.956874 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:41.956874 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:41.956874 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:41.956874 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:41.956874 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:41.956874 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:41.956874 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:41.956874 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:41.956874 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:41.956874 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:41.956874 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:54:41.956874 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:41.959272 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 184/362] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 10 16:54:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:41.960244 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952b654-ff77-4bb9-8876-9449c740ff7e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:42.000877 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:42.047472 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 179/363] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:54:42.052589 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675ccee-c1af-4415-8f95-f164ed703347 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:42.052589 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675ccee-c1af-4415-8f95-f164ed703347 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:42.052589 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675ccee-c1af-4415-8f95-f164ed703347 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:42.052589 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 180/364] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:42.052589 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675ccee-c1af-4415-8f95-f164ed703347 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:42.069529 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:42.110150 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952b654-ff77-4bb9-8876-9449c740ff7e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:42.159320 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be37ce1-44b7-49d3-87e6-63e2ebf28a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:42.159835 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be37ce1-44b7-49d3-87e6-63e2ebf28a46 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:42.160128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be37ce1-44b7-49d3-87e6-63e2ebf28a46 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:42.160369 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be37ce1-44b7-49d3-87e6-63e2ebf28a46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:42.161087 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be37ce1-44b7-49d3-87e6-63e2ebf28a46 None admin] RBAC: Adding query filter params (group.id=0271954fa90a4c699c1864243d5b4040, role.id=a3e5816caacc425c9a6dd381fe0eddf0, scope.project.id=1570ef44f3134332b1fc668345a37479) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:42.161811 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be37ce1-44b7-49d3-87e6-63e2ebf28a46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:42.163277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be37ce1-44b7-49d3-87e6-63e2ebf28a46 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:42.171865 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 185/365] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 10 16:54:41 2022] GET /identity/v3/role_assignments?group.id=0271954fa90a4c699c1864243d5b4040&scope.project.id=1570ef44f3134332b1fc668345a37479&role.id=a3e5816caacc425c9a6dd381fe0eddf0 => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:42.172792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be37ce1-44b7-49d3-87e6-63e2ebf28a46 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:42.204619 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675ccee-c1af-4415-8f95-f164ed703347 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:42.246816 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c03657-3d23-44f5-988d-98cf0c32437a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:42.247836 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c03657-3d23-44f5-988d-98cf0c32437a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:42.248364 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c03657-3d23-44f5-988d-98cf0c32437a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:42.248849 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c03657-3d23-44f5-988d-98cf0c32437a None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:42.249745 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c03657-3d23-44f5-988d-98cf0c32437a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:42.250263 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c03657-3d23-44f5-988d-98cf0c32437a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:42.251944 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c03657-3d23-44f5-988d-98cf0c32437a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:42.262424 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 181/366] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:54:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:42.264045 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fc11a-704d-4694-9a52-a2c167c7d5ab None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:42.264535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fc11a-704d-4694-9a52-a2c167c7d5ab None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:42.265010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fc11a-704d-4694-9a52-a2c167c7d5ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:42.285419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1f9fc11a-704d-4694-9a52-a2c167c7d5ab None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:42.286999 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1f9fc11a-704d-4694-9a52-a2c167c7d5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:42.287697 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:42.302493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be37ce1-44b7-49d3-87e6-63e2ebf28a46 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:42.344920 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b2272-9ec7-4783-9a23-a2654fa7b4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:42.345972 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b2272-9ec7-4783-9a23-a2654fa7b4c3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:42.345972 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b2272-9ec7-4783-9a23-a2654fa7b4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:42.346229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b2272-9ec7-4783-9a23-a2654fa7b4c3 None admin] PATH_INFO: `/v3/projects/793400b224a54914900adad496869b83` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:42.347580 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b2272-9ec7-4783-9a23-a2654fa7b4c3 None admin] RBAC: Authorizing `identity:get_project(project_id=793400b224a54914900adad496869b83)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:42.349456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b2272-9ec7-4783-9a23-a2654fa7b4c3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:42.352532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 186/367] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:42 2022] GET /identity/v3/projects/793400b224a54914900adad496869b83 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:42.353330 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b2272-9ec7-4783-9a23-a2654fa7b4c3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:42.369888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 182/368] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:42.373371 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9fc11a-704d-4694-9a52-a2c167c7d5ab None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:42.377969 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:42.477664 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b2272-9ec7-4783-9a23-a2654fa7b4c3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:42.484867 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9fc11a-704d-4694-9a52-a2c167c7d5ab None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:42.520432 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d72a3f-2b82-4356-b68d-c8218c4b690c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:42.521105 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d72a3f-2b82-4356-b68d-c8218c4b690c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:42.521417 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d72a3f-2b82-4356-b68d-c8218c4b690c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:42.521648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d72a3f-2b82-4356-b68d-c8218c4b690c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:42.529754 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec5c81-63cd-48eb-a752-2f81d1bfe510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:42.530431 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec5c81-63cd-48eb-a752-2f81d1bfe510 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:42.530707 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec5c81-63cd-48eb-a752-2f81d1bfe510 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:42.530975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec5c81-63cd-48eb-a752-2f81d1bfe510 None admin] PATH_INFO: `/v3/groups/0271954fa90a4c699c1864243d5b4040` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:42.535167 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec5c81-63cd-48eb-a752-2f81d1bfe510 None admin] RBAC: Authorizing `identity:get_group(group_id=0271954fa90a4c699c1864243d5b4040)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:42.536954 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec5c81-63cd-48eb-a752-2f81d1bfe510 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:42.539324 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 183/369] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 10 16:54:42 2022] GET /identity/v3/groups/0271954fa90a4c699c1864243d5b4040 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:42.540293 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec5c81-63cd-48eb-a752-2f81d1bfe510 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:42.548934 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:42.603259 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d72a3f-2b82-4356-b68d-c8218c4b690c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:42.605185 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d72a3f-2b82-4356-b68d-c8218c4b690c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:42.647636 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:42.686967 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec5c81-63cd-48eb-a752-2f81d1bfe510 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:42.731788 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3cd177-4883-4df9-8a58-abb571624732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:42.732942 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cd177-4883-4df9-8a58-abb571624732 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:42.733438 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cd177-4883-4df9-8a58-abb571624732 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:42.733920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cd177-4883-4df9-8a58-abb571624732 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:42.738527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 187/370] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:54:42.739424 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d72a3f-2b82-4356-b68d-c8218c4b690c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:42.748688 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3cd177-4883-4df9-8a58-abb571624732 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:42.754428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-fa3cd177-4883-4df9-8a58-abb571624732 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:54:42.754428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:42.754428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:42.754428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:42.754428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:42.754428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:42.754428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:42.754428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:42.754428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:42.754428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:42.754428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:42.754428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:42.754428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:42.754428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:42.754428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:42.754428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:42.755328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:42.755328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:42.755328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:42.755328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:42.755328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:42.755328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:42.755328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:42.755328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:42.755328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:42.755328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:42.755328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:42.755328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:42.755328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:42.755328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:42.755328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:42.755328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:42.756130 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:42.756130 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:42.756130 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:42.756130 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:42.756130 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:42.756130 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:42.756130 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:42.756130 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:42.756130 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:42.756130 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:42.756130 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:54:42.756130 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:54:42.758903 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 184/371] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:54:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:42.759922 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3cd177-4883-4df9-8a58-abb571624732 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:42.803198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:42.852200 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d72a3f-2b82-4356-b68d-c8218c4b690c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:42.876629 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3cd177-4883-4df9-8a58-abb571624732 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:42.888925 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2211d7-6085-4082-8f2d-0d1fcd32754b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:42.889689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2211d7-6085-4082-8f2d-0d1fcd32754b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:42.889799 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2211d7-6085-4082-8f2d-0d1fcd32754b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:42.890065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2211d7-6085-4082-8f2d-0d1fcd32754b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:42.890834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2211d7-6085-4082-8f2d-0d1fcd32754b None admin] RBAC: Adding query filter params (group.id=0271954fa90a4c699c1864243d5b4040, role.id=dfb72dc1f3744c8e957abcc4ac967dac, scope.project.id=793400b224a54914900adad496869b83) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:42.891194 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2211d7-6085-4082-8f2d-0d1fcd32754b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:42.893172 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2211d7-6085-4082-8f2d-0d1fcd32754b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:42.899258 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 188/372] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 10 16:54:42 2022] GET /identity/v3/role_assignments?group.id=0271954fa90a4c699c1864243d5b4040&scope.project.id=793400b224a54914900adad496869b83&role.id=dfb72dc1f3744c8e957abcc4ac967dac => generated 260 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:42.900321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2211d7-6085-4082-8f2d-0d1fcd32754b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:42.917458 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:42.930312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb23064-a354-4735-a9d9-74a4480018cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:42.931360 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb23064-a354-4735-a9d9-74a4480018cf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:42.932023 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb23064-a354-4735-a9d9-74a4480018cf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:42.932656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb23064-a354-4735-a9d9-74a4480018cf None admin] PATH_INFO: `/v3/projects/793400b224a54914900adad496869b83` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:42.934653 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb23064-a354-4735-a9d9-74a4480018cf None admin] RBAC: Authorizing `identity:get_project(project_id=793400b224a54914900adad496869b83)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:42.940604 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb23064-a354-4735-a9d9-74a4480018cf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:42.942863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 185/373] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:42 2022] GET /identity/v3/projects/793400b224a54914900adad496869b83 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:42.948765 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb23064-a354-4735-a9d9-74a4480018cf None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:43.013919 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2211d7-6085-4082-8f2d-0d1fcd32754b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:43.036062 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:43.052788 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682132cb-4e4c-4486-a214-77131d8310ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:43.053398 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682132cb-4e4c-4486-a214-77131d8310ae None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:43.053677 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682132cb-4e4c-4486-a214-77131d8310ae None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:43.053949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682132cb-4e4c-4486-a214-77131d8310ae None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:43.054805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682132cb-4e4c-4486-a214-77131d8310ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:43.055166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682132cb-4e4c-4486-a214-77131d8310ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:43.058198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682132cb-4e4c-4486-a214-77131d8310ae None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:43.074908 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 189/374] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:43.075209 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682132cb-4e4c-4486-a214-77131d8310ae None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:43.115191 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb23064-a354-4735-a9d9-74a4480018cf None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:43.192628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a209c9b4-5acf-4f63-8c0a-11902c1fc55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:43.193193 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682132cb-4e4c-4486-a214-77131d8310ae None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:43.193305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209c9b4-5acf-4f63-8c0a-11902c1fc55a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:43.193544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209c9b4-5acf-4f63-8c0a-11902c1fc55a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:43.193816 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209c9b4-5acf-4f63-8c0a-11902c1fc55a None admin] PATH_INFO: `/v3/groups/0271954fa90a4c699c1864243d5b4040` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:43.199878 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a209c9b4-5acf-4f63-8c0a-11902c1fc55a None admin] RBAC: Authorizing `identity:get_group(group_id=0271954fa90a4c699c1864243d5b4040)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:43.201476 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a209c9b4-5acf-4f63-8c0a-11902c1fc55a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:43.203695 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 186/375] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 10 16:54:42 2022] GET /identity/v3/groups/0271954fa90a4c699c1864243d5b4040 => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:43.209765 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a209c9b4-5acf-4f63-8c0a-11902c1fc55a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:43.242559 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1759224e-6243-4400-9bc9-86506a9fbb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:43.243179 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759224e-6243-4400-9bc9-86506a9fbb1f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:43.243725 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759224e-6243-4400-9bc9-86506a9fbb1f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:43.243996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759224e-6243-4400-9bc9-86506a9fbb1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:43.316787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:43.332840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:43.354829 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759224e-6243-4400-9bc9-86506a9fbb1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:43.356977 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759224e-6243-4400-9bc9-86506a9fbb1f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:43.396314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a209c9b4-5acf-4f63-8c0a-11902c1fc55a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:43.462182 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0ceaa-33fd-40b5-a612-f58aabda548e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:43.462825 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0ceaa-33fd-40b5-a612-f58aabda548e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:43.463105 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0ceaa-33fd-40b5-a612-f58aabda548e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:43.463385 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0ceaa-33fd-40b5-a612-f58aabda548e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:43.464176 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed0ceaa-33fd-40b5-a612-f58aabda548e None admin] RBAC: Adding query filter params (group.id=0271954fa90a4c699c1864243d5b4040, role.id=a3e5816caacc425c9a6dd381fe0eddf0, scope.project.id=793400b224a54914900adad496869b83) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:43.464482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed0ceaa-33fd-40b5-a612-f58aabda548e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:43.466561 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed0ceaa-33fd-40b5-a612-f58aabda548e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:43.473289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 187/376] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 10 16:54:43 2022] GET /identity/v3/role_assignments?group.id=0271954fa90a4c699c1864243d5b4040&scope.project.id=793400b224a54914900adad496869b83&role.id=a3e5816caacc425c9a6dd381fe0eddf0 => generated 260 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:43.525117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 190/377] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:54:43.531914 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0ceaa-33fd-40b5-a612-f58aabda548e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:43.695665 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0ceaa-33fd-40b5-a612-f58aabda548e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:43.728787 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:43.736652 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819c163-cb27-4a59-bac4-cea63049ca96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:43.737299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819c163-cb27-4a59-bac4-cea63049ca96 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:43.737576 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819c163-cb27-4a59-bac4-cea63049ca96 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:43.744870 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819c163-cb27-4a59-bac4-cea63049ca96 None admin] PATH_INFO: `/v3/projects/aff7d0faeed34f6193c691567931a8ff` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:43.744870 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5819c163-cb27-4a59-bac4-cea63049ca96 None admin] RBAC: Authorizing `identity:get_project(project_id=aff7d0faeed34f6193c691567931a8ff)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:43.744870 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5819c163-cb27-4a59-bac4-cea63049ca96 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:43.744870 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 188/378] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:43 2022] GET /identity/v3/projects/aff7d0faeed34f6193c691567931a8ff => generated 344 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:43.751508 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1759224e-6243-4400-9bc9-86506a9fbb1f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:43.816228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce67f0f-81fb-4a2b-aea1-d2a7ad870461 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:43.816530 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce67f0f-81fb-4a2b-aea1-d2a7ad870461 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:43.816780 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce67f0f-81fb-4a2b-aea1-d2a7ad870461 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:43.818135 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 189/379] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:43.825400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c4ec7-afaf-4ff5-92be-28d88897dd0d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:43.825710 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c4ec7-afaf-4ff5-92be-28d88897dd0d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:43.825984 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c4ec7-afaf-4ff5-92be-28d88897dd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:43.846130 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-715c4ec7-afaf-4ff5-92be-28d88897dd0d None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:43.847788 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-715c4ec7-afaf-4ff5-92be-28d88897dd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:43.912035 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:43.931114 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 190/380] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:54:43.954476 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1759224e-6243-4400-9bc9-86506a9fbb1f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:43.960345 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c4ec7-afaf-4ff5-92be-28d88897dd0d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:44.036157 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b143bd5a-acaa-4652-8f52-e4d8d2412a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:44.036804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143bd5a-acaa-4652-8f52-e4d8d2412a20 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:44.037077 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143bd5a-acaa-4652-8f52-e4d8d2412a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:44.037344 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143bd5a-acaa-4652-8f52-e4d8d2412a20 None admin] PATH_INFO: `/v3/groups/3d6df3ac551648d18ab59edbc7a51394` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:44.048859 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b143bd5a-acaa-4652-8f52-e4d8d2412a20 None admin] RBAC: Authorizing `identity:get_group(group_id=3d6df3ac551648d18ab59edbc7a51394)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:44.051914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b143bd5a-acaa-4652-8f52-e4d8d2412a20 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:44.056979 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 191/381] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 10 16:54:43 2022] GET /identity/v3/groups/3d6df3ac551648d18ab59edbc7a51394 => generated 229 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:44.062759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b143bd5a-acaa-4652-8f52-e4d8d2412a20 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:44.129193 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c4ec7-afaf-4ff5-92be-28d88897dd0d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:44.190700 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:44.208173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67dd8bc-9b32-4f98-b684-b99fbeeda5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:44.209169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67dd8bc-9b32-4f98-b684-b99fbeeda5ca None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:44.209671 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67dd8bc-9b32-4f98-b684-b99fbeeda5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:44.210156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67dd8bc-9b32-4f98-b684-b99fbeeda5ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:44.235967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b143bd5a-acaa-4652-8f52-e4d8d2412a20 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:44.237143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67dd8bc-9b32-4f98-b684-b99fbeeda5ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:44.242299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-f67dd8bc-9b32-4f98-b684-b99fbeeda5ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:44.242299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:44.242299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:44.242299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:44.242299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:44.242299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:44.242299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:44.242299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:44.242299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:44.242299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:44.242299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:44.242299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:44.242299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:44.242299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:44.242299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:44.242299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:44.243212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:44.243212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:44.243212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:44.243212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:44.243212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:44.243212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:44.243212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:44.243212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:44.243212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:44.243212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:44.243212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:44.243212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:44.243212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:44.243212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:44.243212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:44.243212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:44.244094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:44.244094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:44.244094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:44.244094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:44.244094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:44.244094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:44.244094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:44.244094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:44.244094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:44.244094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:44.244094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:44.244094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:54:44.250670 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 191/382] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:54:44.253757 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67dd8bc-9b32-4f98-b684-b99fbeeda5ca None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:44.281180 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d117b8dd-0dc6-44de-837e-651afd92f984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:44.281784 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117b8dd-0dc6-44de-837e-651afd92f984 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:44.282067 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117b8dd-0dc6-44de-837e-651afd92f984 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:44.282340 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117b8dd-0dc6-44de-837e-651afd92f984 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:44.283169 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d117b8dd-0dc6-44de-837e-651afd92f984 None admin] RBAC: Adding query filter params (group.id=3d6df3ac551648d18ab59edbc7a51394, role.id=7554b353873942fab506de830188aba0, scope.project.id=aff7d0faeed34f6193c691567931a8ff) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:44.283494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d117b8dd-0dc6-44de-837e-651afd92f984 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:44.290373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d117b8dd-0dc6-44de-837e-651afd92f984 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:44.291363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 192/383] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 10 16:54:44 2022] GET /identity/v3/role_assignments?group.id=3d6df3ac551648d18ab59edbc7a51394&scope.project.id=aff7d0faeed34f6193c691567931a8ff&role.id=7554b353873942fab506de830188aba0 => generated 260 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:44.420318 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67dd8bc-9b32-4f98-b684-b99fbeeda5ca None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:44.460252 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8f5da-6978-4072-8de8-ff35e846443c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:44.460792 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8f5da-6978-4072-8de8-ff35e846443c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:44.461073 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8f5da-6978-4072-8de8-ff35e846443c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:44.461346 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8f5da-6978-4072-8de8-ff35e846443c None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:44.462020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f8f5da-6978-4072-8de8-ff35e846443c None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:44.462323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f8f5da-6978-4072-8de8-ff35e846443c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:44.463740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f8f5da-6978-4072-8de8-ff35e846443c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:44.475120 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 192/384] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:44.480928 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d117b8dd-0dc6-44de-837e-651afd92f984 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:44.495551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:44.589478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d117b8dd-0dc6-44de-837e-651afd92f984 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:44.626334 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4a853-a94a-43e7-b765-dfb605fce799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:44.627014 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4a853-a94a-43e7-b765-dfb605fce799 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:44.627292 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4a853-a94a-43e7-b765-dfb605fce799 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:44.627563 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4a853-a94a-43e7-b765-dfb605fce799 None admin] PATH_INFO: `/v3/groups/0271954fa90a4c699c1864243d5b4040` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:44.628697 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4a853-a94a-43e7-b765-dfb605fce799 None admin] RBAC: Authorizing `identity:get_group(group_id=0271954fa90a4c699c1864243d5b4040)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:44.630282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4a853-a94a-43e7-b765-dfb605fce799 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:44.632406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 193/385] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 10 16:54:44 2022] GET /identity/v3/groups/0271954fa90a4c699c1864243d5b4040 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:44.638931 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8f5da-6978-4072-8de8-ff35e846443c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:44.697660 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:44.751486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8f5da-6978-4072-8de8-ff35e846443c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:44.787425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a90b31-eaee-43f4-9b26-74fdfb797b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:44.788066 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a90b31-eaee-43f4-9b26-74fdfb797b75 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:44.788342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a90b31-eaee-43f4-9b26-74fdfb797b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:44.788612 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a90b31-eaee-43f4-9b26-74fdfb797b75 None admin] PATH_INFO: `/v3/projects/1570ef44f3134332b1fc668345a37479` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:44.789752 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a90b31-eaee-43f4-9b26-74fdfb797b75 None admin] RBAC: Authorizing `identity:get_project(project_id=1570ef44f3134332b1fc668345a37479)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:44.791257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a90b31-eaee-43f4-9b26-74fdfb797b75 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:44.793302 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 193/386] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:44 2022] GET /identity/v3/projects/1570ef44f3134332b1fc668345a37479 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:44.799130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4a853-a94a-43e7-b765-dfb605fce799 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:44.836854 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719a3e9-6b8d-4e32-b491-294415fb3f41 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:44.837158 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719a3e9-6b8d-4e32-b491-294415fb3f41 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:44.837398 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719a3e9-6b8d-4e32-b491-294415fb3f41 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:44.838953 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 194/387] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:54:44.844086 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb77b8-bf0a-4f83-b38b-0f0914795298 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:44.844374 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb77b8-bf0a-4f83-b38b-0f0914795298 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:44.844649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb77b8-bf0a-4f83-b38b-0f0914795298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:44.870660 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-76fb77b8-bf0a-4f83-b38b-0f0914795298 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:44.873007 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-76fb77b8-bf0a-4f83-b38b-0f0914795298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:44.950391 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:44.952735 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 195/388] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:54:44.987050 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb77b8-bf0a-4f83-b38b-0f0914795298 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:44.996653 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4a853-a94a-43e7-b765-dfb605fce799 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:45.059375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:45.068403 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5f6d0-324d-428f-8a7a-7f4a1ce24dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:45.069392 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5f6d0-324d-428f-8a7a-7f4a1ce24dc0 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:45.089423 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5f6d0-324d-428f-8a7a-7f4a1ce24dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:45.089423 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5f6d0-324d-428f-8a7a-7f4a1ce24dc0 None admin] PATH_INFO: `/v3/projects/1570ef44f3134332b1fc668345a37479/groups/0271954fa90a4c699c1864243d5b4040/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:45.089423 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5f6d0-324d-428f-8a7a-7f4a1ce24dc0 None admin] RBAC: Authorizing `identity:create_grant(project_id=1570ef44f3134332b1fc668345a37479, group_id=0271954fa90a4c699c1864243d5b4040, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:45.089423 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5f6d0-324d-428f-8a7a-7f4a1ce24dc0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:45.140139 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb77b8-bf0a-4f83-b38b-0f0914795298 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:45.165293 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 194/389] 10.222.0.45 () {58 vars in 1442 bytes} [Wed Aug 10 16:54:44 2022] PUT /identity/v3/projects/1570ef44f3134332b1fc668345a37479/groups/0271954fa90a4c699c1864243d5b4040/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:45.203006 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb85f5-cc34-4ffd-ac83-3aabcfcd1879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:45.203998 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb85f5-cc34-4ffd-ac83-3aabcfcd1879 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:45.204528 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb85f5-cc34-4ffd-ac83-3aabcfcd1879 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:45.205102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb85f5-cc34-4ffd-ac83-3aabcfcd1879 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:45.223760 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb85f5-cc34-4ffd-ac83-3aabcfcd1879 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:45.229503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-39fb85f5-cc34-4ffd-ac83-3aabcfcd1879 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:54:45.229503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:45.229503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:45.229503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:45.229503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:45.229503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:45.229503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:45.229503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:45.229503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:45.229503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:45.229503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:45.229503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:45.229503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:45.229503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:45.229503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:45.229503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:45.230479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:45.230479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:45.230479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:45.230479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:45.230479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:45.230479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:45.230479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:45.230479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:45.230479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:45.230479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:45.230479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:45.230479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:45.230479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:45.230479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:45.230479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:45.230479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:45.231305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:45.231305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:45.231305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:45.231305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:45.231305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:45.231305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:45.231305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:45.231305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:45.231305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:45.231305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:45.231305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:54:45.231305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:54:45.234111 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 196/390] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 10 16:54:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:54:45.239823 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5f6d0-324d-428f-8a7a-7f4a1ce24dc0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:45.351938 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5f6d0-324d-428f-8a7a-7f4a1ce24dc0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:45.392453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d301fe-982e-4bbb-82d8-e391023afa11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:45.393414 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d301fe-982e-4bbb-82d8-e391023afa11 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:45.393923 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d301fe-982e-4bbb-82d8-e391023afa11 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:45.394488 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d301fe-982e-4bbb-82d8-e391023afa11 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:45.395361 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d301fe-982e-4bbb-82d8-e391023afa11 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:45.395868 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d301fe-982e-4bbb-82d8-e391023afa11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:45.397444 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d301fe-982e-4bbb-82d8-e391023afa11 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:45.406962 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:45.412115 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 195/391] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:54:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:45.417651 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb85f5-cc34-4ffd-ac83-3aabcfcd1879 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:45.538602 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb85f5-cc34-4ffd-ac83-3aabcfcd1879 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:45.551895 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:45.585786 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba36442-57f9-4a6a-ada7-bb6146dd3d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:45.587290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba36442-57f9-4a6a-ada7-bb6146dd3d0a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:45.587874 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba36442-57f9-4a6a-ada7-bb6146dd3d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:45.588635 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba36442-57f9-4a6a-ada7-bb6146dd3d0a None admin] PATH_INFO: `/v3/groups/0271954fa90a4c699c1864243d5b4040` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:45.590369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba36442-57f9-4a6a-ada7-bb6146dd3d0a None admin] RBAC: Authorizing `identity:get_group(group_id=0271954fa90a4c699c1864243d5b4040)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:45.591842 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba36442-57f9-4a6a-ada7-bb6146dd3d0a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:45.593766 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 197/392] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 10 16:54:45 2022] GET /identity/v3/groups/0271954fa90a4c699c1864243d5b4040 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:45.599008 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d301fe-982e-4bbb-82d8-e391023afa11 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:45.715523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d301fe-982e-4bbb-82d8-e391023afa11 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:45.783110 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4c43a-31aa-4fd0-9039-14c541b275ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:45.783755 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4c43a-31aa-4fd0-9039-14c541b275ed None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:45.784025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4c43a-31aa-4fd0-9039-14c541b275ed None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:45.784288 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4c43a-31aa-4fd0-9039-14c541b275ed None admin] PATH_INFO: `/v3/projects/1570ef44f3134332b1fc668345a37479` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:45.785479 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4c43a-31aa-4fd0-9039-14c541b275ed None admin] RBAC: Authorizing `identity:get_project(project_id=1570ef44f3134332b1fc668345a37479)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:45.786939 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4c43a-31aa-4fd0-9039-14c541b275ed None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:45.788896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 196/393] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:45 2022] GET /identity/v3/projects/1570ef44f3134332b1fc668345a37479 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:45.794835 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba36442-57f9-4a6a-ada7-bb6146dd3d0a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:45.915950 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a9cfa-56c2-4b08-a6b2-16149b455091 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:45.916128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a9cfa-56c2-4b08-a6b2-16149b455091 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:45.916404 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a9cfa-56c2-4b08-a6b2-16149b455091 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:45.917982 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 197/394] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:45.923246 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bdfda-9f99-4c5c-8be7-326d14e403a7 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:45.923530 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bdfda-9f99-4c5c-8be7-326d14e403a7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:45.923811 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bdfda-9f99-4c5c-8be7-326d14e403a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:45.947077 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba36442-57f9-4a6a-ada7-bb6146dd3d0a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:45.974547 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ac5bdfda-9f99-4c5c-8be7-326d14e403a7 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:45.975754 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ac5bdfda-9f99-4c5c-8be7-326d14e403a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:45.995296 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a93e84-6f83-4cc7-8700-d98b4a1def54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:45.995996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a93e84-6f83-4cc7-8700-d98b4a1def54 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:45.996273 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a93e84-6f83-4cc7-8700-d98b4a1def54 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:45.996538 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a93e84-6f83-4cc7-8700-d98b4a1def54 None admin] PATH_INFO: `/v3/projects/1570ef44f3134332b1fc668345a37479/groups/0271954fa90a4c699c1864243d5b4040/roles/a3e5816caacc425c9a6dd381fe0eddf0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:46.002749 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a93e84-6f83-4cc7-8700-d98b4a1def54 None admin] RBAC: Authorizing `identity:create_grant(project_id=1570ef44f3134332b1fc668345a37479, group_id=0271954fa90a4c699c1864243d5b4040, role_id=a3e5816caacc425c9a6dd381fe0eddf0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:46.004219 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a93e84-6f83-4cc7-8700-d98b4a1def54 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:46.062835 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:46.074539 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 198/395] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:54:46.093150 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5bdfda-9f99-4c5c-8be7-326d14e403a7 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:46.188771 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 198/396] 10.222.0.45 () {58 vars in 1442 bytes} [Wed Aug 10 16:54:45 2022] PUT /identity/v3/projects/1570ef44f3134332b1fc668345a37479/groups/0271954fa90a4c699c1864243d5b4040/roles/a3e5816caacc425c9a6dd381fe0eddf0 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:46.190498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7d2fe-55de-43f0-a49b-70d5f3f406da None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:46.190498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7d2fe-55de-43f0-a49b-70d5f3f406da None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:46.190781 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7d2fe-55de-43f0-a49b-70d5f3f406da None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:46.192404 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 199/397] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:46.201118 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48867dd2-8156-47fa-a50b-bb1c6c45d3ae None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:46.201419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48867dd2-8156-47fa-a50b-bb1c6c45d3ae None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:46.201829 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48867dd2-8156-47fa-a50b-bb1c6c45d3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:46.209862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5bdfda-9f99-4c5c-8be7-326d14e403a7 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:46.240687 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-48867dd2-8156-47fa-a50b-bb1c6c45d3ae None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:46.241898 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-48867dd2-8156-47fa-a50b-bb1c6c45d3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:46.302395 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5266fb-a08a-4e56-85c7-f9c822c4a230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:46.303020 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5266fb-a08a-4e56-85c7-f9c822c4a230 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:46.303297 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5266fb-a08a-4e56-85c7-f9c822c4a230 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:46.303569 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5266fb-a08a-4e56-85c7-f9c822c4a230 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:46.315510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:46.326782 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5266fb-a08a-4e56-85c7-f9c822c4a230 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:46.332884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-4c5266fb-a08a-4e56-85c7-f9c822c4a230 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:46.332884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:46.332884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:46.332884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:46.332884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:46.332884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:46.332884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:46.332884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:46.332884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:46.332884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:46.332884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:46.332884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:46.332884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:46.332884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:46.332884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:46.332884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:46.333822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:46.333822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:46.333822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:46.333822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:46.333822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:46.333822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:46.333822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:46.333822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:46.333822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:46.333822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:46.333822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:46.333822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:46.333822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:46.333822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:46.333822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:46.333822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:46.334701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:46.334701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:46.334701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:46.334701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:46.334701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:46.334701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:46.334701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:46.334701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:46.334701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:46.334701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:46.334701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:46.334701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:46.334701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 199/398] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:54:46.341785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5266fb-a08a-4e56-85c7-f9c822c4a230 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:46.394255 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 200/399] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:46.431542 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48867dd2-8156-47fa-a50b-bb1c6c45d3ae None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:46.437804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:46.504387 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5266fb-a08a-4e56-85c7-f9c822c4a230 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:46.546662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e8cb0-2675-41d9-bf32-8349a2f24733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:46.547276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e8cb0-2675-41d9-bf32-8349a2f24733 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:46.547554 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e8cb0-2675-41d9-bf32-8349a2f24733 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:46.547826 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e8cb0-2675-41d9-bf32-8349a2f24733 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:46.548761 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2e8cb0-2675-41d9-bf32-8349a2f24733 None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:46.548965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2e8cb0-2675-41d9-bf32-8349a2f24733 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:46.550279 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2e8cb0-2675-41d9-bf32-8349a2f24733 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:46.560679 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 200/400] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:46.567316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e8cb0-2675-41d9-bf32-8349a2f24733 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:46.578933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48867dd2-8156-47fa-a50b-bb1c6c45d3ae None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:46.633197 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce5529-45f9-450a-bbff-0c6b349098fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:46.634553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce5529-45f9-450a-bbff-0c6b349098fd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:46.635052 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce5529-45f9-450a-bbff-0c6b349098fd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:46.635740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce5529-45f9-450a-bbff-0c6b349098fd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:46.651930 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce5529-45f9-450a-bbff-0c6b349098fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:46.659429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-87ce5529-45f9-450a-bbff-0c6b349098fd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:54:46.659429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:46.659429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:46.659429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:46.659429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:46.659429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:46.659429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:46.659429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:46.659429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:46.659429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:46.659429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:46.659429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:46.659429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:46.659429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:46.659429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:46.659429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:46.670140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:46.670140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:46.670140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:46.670140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:46.670140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:46.670140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:46.670140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:46.670140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:46.670140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:46.670140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:46.670140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:46.670140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:46.670140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:46.670140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:46.670140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:46.670140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:46.671014 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:46.671014 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:46.671014 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:46.671014 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:46.671014 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:46.671014 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:46.671014 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:46.671014 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:46.671014 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:46.671014 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:46.671014 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:54:46.671014 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:54:46.671014 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 201/401] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 10 16:54:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:54:46.671014 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce5529-45f9-450a-bbff-0c6b349098fd None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:46.714411 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:46.724201 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e8cb0-2675-41d9-bf32-8349a2f24733 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:46.771933 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:46.800117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c3f0a-bb8e-4399-9b31-148fcccc7765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:46.803317 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c3f0a-bb8e-4399-9b31-148fcccc7765 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:46.803317 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c3f0a-bb8e-4399-9b31-148fcccc7765 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:46.803317 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c3f0a-bb8e-4399-9b31-148fcccc7765 None admin] PATH_INFO: `/v3/groups/0271954fa90a4c699c1864243d5b4040` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:46.804116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c3f0a-bb8e-4399-9b31-148fcccc7765 None admin] RBAC: Authorizing `identity:get_group(group_id=0271954fa90a4c699c1864243d5b4040)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:46.810050 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c3f0a-bb8e-4399-9b31-148fcccc7765 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:46.812045 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 201/402] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 10 16:54:46 2022] GET /identity/v3/groups/0271954fa90a4c699c1864243d5b4040 => generated 226 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:46.818073 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c3f0a-bb8e-4399-9b31-148fcccc7765 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:46.859005 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce5529-45f9-450a-bbff-0c6b349098fd None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:46.902549 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d370f4e-9630-4572-82a6-01bc7cb4a921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:46.902549 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d370f4e-9630-4572-82a6-01bc7cb4a921 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:46.902549 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d370f4e-9630-4572-82a6-01bc7cb4a921 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:46.902549 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d370f4e-9630-4572-82a6-01bc7cb4a921 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:46.902980 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d370f4e-9630-4572-82a6-01bc7cb4a921 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:46.902980 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d370f4e-9630-4572-82a6-01bc7cb4a921 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:46.904314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d370f4e-9630-4572-82a6-01bc7cb4a921 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:46.913441 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 202/403] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:54:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:46.923650 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d370f4e-9630-4572-82a6-01bc7cb4a921 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:46.997996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c3f0a-bb8e-4399-9b31-148fcccc7765 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:47.055567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5156a-ada0-4677-98dc-ae30de5c6275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:47.056116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5156a-ada0-4677-98dc-ae30de5c6275 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:47.056391 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5156a-ada0-4677-98dc-ae30de5c6275 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:47.056658 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5156a-ada0-4677-98dc-ae30de5c6275 None admin] PATH_INFO: `/v3/projects/793400b224a54914900adad496869b83` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:47.057851 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de5156a-ada0-4677-98dc-ae30de5c6275 None admin] RBAC: Authorizing `identity:get_project(project_id=793400b224a54914900adad496869b83)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:47.059946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de5156a-ada0-4677-98dc-ae30de5c6275 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:47.061998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 202/404] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:46 2022] GET /identity/v3/projects/793400b224a54914900adad496869b83 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:47.067716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5156a-ada0-4677-98dc-ae30de5c6275 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:47.090316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:47.108563 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d370f4e-9630-4572-82a6-01bc7cb4a921 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:47.160323 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:47.174024 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe298f6-9a40-44f4-aea9-62e740a661d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:47.174686 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe298f6-9a40-44f4-aea9-62e740a661d9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:47.174961 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe298f6-9a40-44f4-aea9-62e740a661d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:47.175226 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe298f6-9a40-44f4-aea9-62e740a661d9 None admin] PATH_INFO: `/v3/groups/0271954fa90a4c699c1864243d5b4040` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:47.177238 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe298f6-9a40-44f4-aea9-62e740a661d9 None admin] RBAC: Authorizing `identity:get_group(group_id=0271954fa90a4c699c1864243d5b4040)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:47.178726 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe298f6-9a40-44f4-aea9-62e740a661d9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:47.180630 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 203/405] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 10 16:54:46 2022] GET /identity/v3/groups/0271954fa90a4c699c1864243d5b4040 => generated 226 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:47.188772 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe298f6-9a40-44f4-aea9-62e740a661d9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:47.228962 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5156a-ada0-4677-98dc-ae30de5c6275 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:47.276531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab788ff-724b-4d40-95b6-2403808e0a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:47.277278 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab788ff-724b-4d40-95b6-2403808e0a9b None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:47.278026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab788ff-724b-4d40-95b6-2403808e0a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:47.278026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab788ff-724b-4d40-95b6-2403808e0a9b None admin] PATH_INFO: `/v3/projects/793400b224a54914900adad496869b83/groups/0271954fa90a4c699c1864243d5b4040/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:47.280430 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab788ff-724b-4d40-95b6-2403808e0a9b None admin] RBAC: Authorizing `identity:create_grant(project_id=793400b224a54914900adad496869b83, group_id=0271954fa90a4c699c1864243d5b4040, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:47.282566 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab788ff-724b-4d40-95b6-2403808e0a9b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:47.325228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe298f6-9a40-44f4-aea9-62e740a661d9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:47.362330 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b73c80-d805-481c-8822-6a674afcb6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:47.362575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b73c80-d805-481c-8822-6a674afcb6d7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:47.362819 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b73c80-d805-481c-8822-6a674afcb6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:47.363124 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b73c80-d805-481c-8822-6a674afcb6d7 None admin] PATH_INFO: `/v3/projects/793400b224a54914900adad496869b83` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:47.364255 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b73c80-d805-481c-8822-6a674afcb6d7 None admin] RBAC: Authorizing `identity:get_project(project_id=793400b224a54914900adad496869b83)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:47.365689 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b73c80-d805-481c-8822-6a674afcb6d7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:47.372274 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 204/406] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:47 2022] GET /identity/v3/projects/793400b224a54914900adad496869b83 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:47.372274 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b260ce7-d75f-4441-b287-2b92075393e8 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:47.372274 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b260ce7-d75f-4441-b287-2b92075393e8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:47.372274 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b260ce7-d75f-4441-b287-2b92075393e8 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:47.372274 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 205/407] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:47.375806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b260ce7-d75f-4441-b287-2b92075393e8 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:47.412282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:47.435228 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 203/408] 10.222.0.45 () {58 vars in 1442 bytes} [Wed Aug 10 16:54:47 2022] PUT /identity/v3/projects/793400b224a54914900adad496869b83/groups/0271954fa90a4c699c1864243d5b4040/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:54:47.437931 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3bdc8-223b-4a91-9be9-50131f45d3dc None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:47.442663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3bdc8-223b-4a91-9be9-50131f45d3dc None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:47.443029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3bdc8-223b-4a91-9be9-50131f45d3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:47.478089 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1bf3bdc8-223b-4a91-9be9-50131f45d3dc None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:47.483399 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1bf3bdc8-223b-4a91-9be9-50131f45d3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:47.563605 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b260ce7-d75f-4441-b287-2b92075393e8 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:47.619807 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 204/409] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:47.643202 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837c3fe-6ce6-4a83-8d2c-e485f8ded77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:47.643888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837c3fe-6ce6-4a83-8d2c-e485f8ded77b None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:47.644164 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837c3fe-6ce6-4a83-8d2c-e485f8ded77b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:47.644437 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837c3fe-6ce6-4a83-8d2c-e485f8ded77b None admin] PATH_INFO: `/v3/projects/793400b224a54914900adad496869b83/groups/0271954fa90a4c699c1864243d5b4040/roles/a3e5816caacc425c9a6dd381fe0eddf0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:47.646760 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf3bdc8-223b-4a91-9be9-50131f45d3dc None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:47.647694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837c3fe-6ce6-4a83-8d2c-e485f8ded77b None admin] RBAC: Authorizing `identity:create_grant(project_id=793400b224a54914900adad496869b83, group_id=0271954fa90a4c699c1864243d5b4040, role_id=a3e5816caacc425c9a6dd381fe0eddf0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:47.649247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837c3fe-6ce6-4a83-8d2c-e485f8ded77b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:47.704395 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:47.740672 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 206/410] 10.222.0.45 () {58 vars in 1442 bytes} [Wed Aug 10 16:54:47 2022] PUT /identity/v3/projects/793400b224a54914900adad496869b83/groups/0271954fa90a4c699c1864243d5b4040/roles/a3e5816caacc425c9a6dd381fe0eddf0 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:54:47.773314 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf3bdc8-223b-4a91-9be9-50131f45d3dc None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:47.818656 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608c019-3965-4d85-b56a-2336202dfbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:47.819295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608c019-3965-4d85-b56a-2336202dfbc5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:47.819569 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608c019-3965-4d85-b56a-2336202dfbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:47.819837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608c019-3965-4d85-b56a-2336202dfbc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:47.837800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608c019-3965-4d85-b56a-2336202dfbc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:47.842966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-9608c019-3965-4d85-b56a-2336202dfbc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:54:47.842966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:47.842966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:47.842966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:47.842966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:47.842966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:47.842966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:47.842966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:47.842966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:47.842966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:47.842966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:47.842966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:47.842966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:47.842966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:47.842966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:47.842966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:47.844034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:47.844034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:47.844034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:47.844034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:47.844034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:47.844034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:47.844034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:47.844034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:47.844034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:47.844034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:47.844034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:47.844034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:47.844034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:47.844034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:47.844034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:47.844034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:47.844834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:47.844834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:47.844834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:47.844834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:47.844834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:47.844834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:47.844834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:47.844834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:47.844834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:47.844834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:47.844834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:54:47.844834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:47.844834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 205/411] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:54:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:54:47.850166 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837c3fe-6ce6-4a83-8d2c-e485f8ded77b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:47.857201 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe132328-1136-41ab-a87f-b9ed070744ba None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:47.857510 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe132328-1136-41ab-a87f-b9ed070744ba None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:47.857729 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe132328-1136-41ab-a87f-b9ed070744ba None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:47.859406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 206/412] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:54:47.864879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccae1f-7dbd-4cb6-a69c-b8131338e606 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:47.865149 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccae1f-7dbd-4cb6-a69c-b8131338e606 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:47.865400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccae1f-7dbd-4cb6-a69c-b8131338e606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:47.890031 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-96ccae1f-7dbd-4cb6-a69c-b8131338e606 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:47.894441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-96ccae1f-7dbd-4cb6-a69c-b8131338e606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:47.995475 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:48.002529 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 207/413] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:54:48.013140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837c3fe-6ce6-4a83-8d2c-e485f8ded77b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:48.038613 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ccae1f-7dbd-4cb6-a69c-b8131338e606 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:48.057666 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746fa35d-f291-49ac-a78b-8e09e3d2bb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:48.060035 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fa35d-f291-49ac-a78b-8e09e3d2bb5a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:48.060425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fa35d-f291-49ac-a78b-8e09e3d2bb5a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:48.060804 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fa35d-f291-49ac-a78b-8e09e3d2bb5a None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:48.061592 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746fa35d-f291-49ac-a78b-8e09e3d2bb5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:48.061949 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746fa35d-f291-49ac-a78b-8e09e3d2bb5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:48.063701 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746fa35d-f291-49ac-a78b-8e09e3d2bb5a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:48.078490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 207/414] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:48.081523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746fa35d-f291-49ac-a78b-8e09e3d2bb5a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:48.095506 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:48.196699 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746fa35d-f291-49ac-a78b-8e09e3d2bb5a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:48.197033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ccae1f-7dbd-4cb6-a69c-b8131338e606 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:48.233830 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff4ed6-4368-4ac2-a58d-582614566a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:48.234968 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff4ed6-4368-4ac2-a58d-582614566a22 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:48.235453 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff4ed6-4368-4ac2-a58d-582614566a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:48.235915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff4ed6-4368-4ac2-a58d-582614566a22 None admin] PATH_INFO: `/v3/groups/3d6df3ac551648d18ab59edbc7a51394` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:48.237229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff4ed6-4368-4ac2-a58d-582614566a22 None admin] RBAC: Authorizing `identity:get_group(group_id=3d6df3ac551648d18ab59edbc7a51394)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:48.238903 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff4ed6-4368-4ac2-a58d-582614566a22 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:48.240984 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 208/415] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 10 16:54:48 2022] GET /identity/v3/groups/3d6df3ac551648d18ab59edbc7a51394 => generated 229 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:48.246681 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff4ed6-4368-4ac2-a58d-582614566a22 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:48.264143 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb20845-ec39-4857-8200-43cba2e61c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:48.264786 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb20845-ec39-4857-8200-43cba2e61c9b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:48.265067 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb20845-ec39-4857-8200-43cba2e61c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:48.265345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb20845-ec39-4857-8200-43cba2e61c9b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:48.267348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:48.279439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb20845-ec39-4857-8200-43cba2e61c9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:48.284527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-afb20845-ec39-4857-8200-43cba2e61c9b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:48.284527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:48.284527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:48.284527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:48.284527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:48.284527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:48.284527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:48.284527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:48.284527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:48.284527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:48.284527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:48.284527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:48.284527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:48.284527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:48.284527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:48.284527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:48.291087 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:48.291087 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:48.291087 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:48.291087 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:48.291087 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:48.291087 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:48.291087 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:48.291087 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:48.291087 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:48.291087 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:48.291087 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:48.291087 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:48.291087 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:48.291087 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:48.291087 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:48.291087 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:48.291884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:48.291884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:48.291884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:48.291884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:48.291884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:48.291884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:48.291884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:48.291884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:48.291884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:48.291884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:48.291884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:48.291884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:48.291884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 208/416] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:54:48.293671 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb20845-ec39-4857-8200-43cba2e61c9b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:48.362248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff4ed6-4368-4ac2-a58d-582614566a22 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:48.437374 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d1229-2c79-48c6-ad1b-39ff7395ccf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:48.438131 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d1229-2c79-48c6-ad1b-39ff7395ccf6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:48.438437 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d1229-2c79-48c6-ad1b-39ff7395ccf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:48.438713 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d1229-2c79-48c6-ad1b-39ff7395ccf6 None admin] PATH_INFO: `/v3/projects/aff7d0faeed34f6193c691567931a8ff` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:48.440999 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d1229-2c79-48c6-ad1b-39ff7395ccf6 None admin] RBAC: Authorizing `identity:get_project(project_id=aff7d0faeed34f6193c691567931a8ff)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:48.442498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d1229-2c79-48c6-ad1b-39ff7395ccf6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:48.444681 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 209/417] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:48 2022] GET /identity/v3/projects/aff7d0faeed34f6193c691567931a8ff => generated 344 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:48.450446 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:48.451574 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d1229-2c79-48c6-ad1b-39ff7395ccf6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:48.456052 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb20845-ec39-4857-8200-43cba2e61c9b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:48.503630 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fa0c4-ece9-4134-999f-dcec3bd67b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:48.504238 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fa0c4-ece9-4134-999f-dcec3bd67b7a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:48.504812 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fa0c4-ece9-4134-999f-dcec3bd67b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:48.505094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fa0c4-ece9-4134-999f-dcec3bd67b7a None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:48.505769 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fa0c4-ece9-4134-999f-dcec3bd67b7a None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:48.506070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fa0c4-ece9-4134-999f-dcec3bd67b7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:48.507489 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fa0c4-ece9-4134-999f-dcec3bd67b7a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:48.524944 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 209/418] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:48.534373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fa0c4-ece9-4134-999f-dcec3bd67b7a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:48.594382 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:48.631420 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d1229-2c79-48c6-ad1b-39ff7395ccf6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:48.691337 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77654255-7d12-4e16-8f39-23c4c37403bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:48.692005 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77654255-7d12-4e16-8f39-23c4c37403bb None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:48.692284 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77654255-7d12-4e16-8f39-23c4c37403bb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:48.692557 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77654255-7d12-4e16-8f39-23c4c37403bb None admin] PATH_INFO: `/v3/projects/aff7d0faeed34f6193c691567931a8ff/groups/3d6df3ac551648d18ab59edbc7a51394/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:48.695800 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77654255-7d12-4e16-8f39-23c4c37403bb None admin] RBAC: Authorizing `identity:create_grant(project_id=aff7d0faeed34f6193c691567931a8ff, group_id=3d6df3ac551648d18ab59edbc7a51394, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:48.697230 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77654255-7d12-4e16-8f39-23c4c37403bb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:48.717581 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fa0c4-ece9-4134-999f-dcec3bd67b7a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:48.753509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663b123-7728-45f4-808b-f1325ef7ed66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:48.754111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663b123-7728-45f4-808b-f1325ef7ed66 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:48.754414 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663b123-7728-45f4-808b-f1325ef7ed66 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:48.754686 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663b123-7728-45f4-808b-f1325ef7ed66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:48.781396 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 210/419] 10.222.0.45 () {58 vars in 1442 bytes} [Wed Aug 10 16:54:48 2022] PUT /identity/v3/projects/aff7d0faeed34f6193c691567931a8ff/groups/3d6df3ac551648d18ab59edbc7a51394/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:54:48.782864 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ebae2-d6e9-4eb3-bcaf-23c02f6185ff None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:48.783151 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ebae2-d6e9-4eb3-bcaf-23c02f6185ff None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:48.783868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ebae2-d6e9-4eb3-bcaf-23c02f6185ff None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:48.789112 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 211/420] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:48.791404 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:48.793835 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f2992-b3ff-4da8-ae1f-696b9cea37aa None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:48.794564 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f2992-b3ff-4da8-ae1f-696b9cea37aa None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:48.795064 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f2992-b3ff-4da8-ae1f-696b9cea37aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:48.818056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c17f2992-b3ff-4da8-ae1f-696b9cea37aa None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:48.820045 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c17f2992-b3ff-4da8-ae1f-696b9cea37aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:48.845901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663b123-7728-45f4-808b-f1325ef7ed66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:48.847488 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663b123-7728-45f4-808b-f1325ef7ed66 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:48.988691 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 212/421] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:49.003944 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 210/422] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 16:54:49.010673 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f2992-b3ff-4da8-ae1f-696b9cea37aa None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:49.020742 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663b123-7728-45f4-808b-f1325ef7ed66 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:49.121762 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:49.127577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:49.144878 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f2992-b3ff-4da8-ae1f-696b9cea37aa None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:49.186537 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b0aa9-2fa8-4a5b-b77d-8e8c241cb97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:49.187351 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b0aa9-2fa8-4a5b-b77d-8e8c241cb97c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:49.188040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b0aa9-2fa8-4a5b-b77d-8e8c241cb97c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:49.188437 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b0aa9-2fa8-4a5b-b77d-8e8c241cb97c None admin] PATH_INFO: `/v3/projects/1570ef44f3134332b1fc668345a37479` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:49.189829 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663b123-7728-45f4-808b-f1325ef7ed66 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:49.190635 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b0aa9-2fa8-4a5b-b77d-8e8c241cb97c None admin] RBAC: Authorizing `identity:get_project(project_id=1570ef44f3134332b1fc668345a37479)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:49.197722 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b0aa9-2fa8-4a5b-b77d-8e8c241cb97c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:49.198093 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 213/423] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:49 2022] GET /identity/v3/projects/1570ef44f3134332b1fc668345a37479 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:49.201522 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b0aa9-2fa8-4a5b-b77d-8e8c241cb97c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:49.314905 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b0aa9-2fa8-4a5b-b77d-8e8c241cb97c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:49.331115 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d46d1d-1ca3-494c-b4a6-6a46d5301405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:49.331750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d46d1d-1ca3-494c-b4a6-6a46d5301405 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:49.334377 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d46d1d-1ca3-494c-b4a6-6a46d5301405 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:49.334377 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d46d1d-1ca3-494c-b4a6-6a46d5301405 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:49.353001 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e204618f-2ba1-4085-9336-48b8d750e459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:49.353635 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204618f-2ba1-4085-9336-48b8d750e459 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:49.357408 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204618f-2ba1-4085-9336-48b8d750e459 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:49.357408 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204618f-2ba1-4085-9336-48b8d750e459 None admin] PATH_INFO: `/v3/groups/0271954fa90a4c699c1864243d5b4040` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:49.357408 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e204618f-2ba1-4085-9336-48b8d750e459 None admin] RBAC: Authorizing `identity:get_group(group_id=0271954fa90a4c699c1864243d5b4040)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:49.357408 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e204618f-2ba1-4085-9336-48b8d750e459 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:49.362613 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 214/424] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 10 16:54:49 2022] GET /identity/v3/groups/0271954fa90a4c699c1864243d5b4040 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:49.364740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e204618f-2ba1-4085-9336-48b8d750e459 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:49.378959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:49.394383 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d46d1d-1ca3-494c-b4a6-6a46d5301405 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:49.396439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-76d46d1d-1ca3-494c-b4a6-6a46d5301405 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:54:49.396439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:49.396439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:49.396439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:49.396439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:49.396439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:49.396439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:49.396439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:49.396439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:49.396439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:49.396439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:49.396439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:49.396439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:49.396439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:49.396439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:49.396439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:49.399251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:49.399251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:49.399251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:49.399251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:49.399251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:49.399251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:49.399251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:49.399251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:49.399251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:49.399251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:49.399251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:49.399251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:49.399251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:49.399251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:49.399251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:49.399251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:49.400519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:49.400519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:49.400519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:49.400519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:49.400519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:49.400519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:49.400519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:49.400519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:49.400519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:49.400519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:49.400519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:54:49.400519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:49.406938 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 211/425] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 10 16:54:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:54:49.412753 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d46d1d-1ca3-494c-b4a6-6a46d5301405 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:49.475424 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e204618f-2ba1-4085-9336-48b8d750e459 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:49.534383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5502b33-9e43-48fa-b590-1b0f60aebb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:49.535143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5502b33-9e43-48fa-b590-1b0f60aebb5d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:49.535425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5502b33-9e43-48fa-b590-1b0f60aebb5d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:49.535796 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5502b33-9e43-48fa-b590-1b0f60aebb5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:49.536529 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5502b33-9e43-48fa-b590-1b0f60aebb5d None admin] RBAC: Adding query filter params (group.id=0271954fa90a4c699c1864243d5b4040, role.id=dfb72dc1f3744c8e957abcc4ac967dac, scope.project.id=1570ef44f3134332b1fc668345a37479) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:49.536833 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5502b33-9e43-48fa-b590-1b0f60aebb5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:49.538266 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5502b33-9e43-48fa-b590-1b0f60aebb5d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:49.554463 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 215/426] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 10 16:54:49 2022] GET /identity/v3/role_assignments?group.id=0271954fa90a4c699c1864243d5b4040&scope.project.id=1570ef44f3134332b1fc668345a37479&role.id=dfb72dc1f3744c8e957abcc4ac967dac => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:49.576495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d46d1d-1ca3-494c-b4a6-6a46d5301405 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:49.627529 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:49.670774 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e7d31-11ef-4ec3-9146-fac3bbfe6c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:49.671757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e7d31-11ef-4ec3-9146-fac3bbfe6c01 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:49.672294 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e7d31-11ef-4ec3-9146-fac3bbfe6c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:49.672804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e7d31-11ef-4ec3-9146-fac3bbfe6c01 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:49.673682 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e7d31-11ef-4ec3-9146-fac3bbfe6c01 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:49.674187 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e7d31-11ef-4ec3-9146-fac3bbfe6c01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:49.675782 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e7d31-11ef-4ec3-9146-fac3bbfe6c01 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:49.701475 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 212/427] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:54:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:49.709091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5502b33-9e43-48fa-b590-1b0f60aebb5d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:49.834965 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:49.859878 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5502b33-9e43-48fa-b590-1b0f60aebb5d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:49.900199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c21e7-52ca-4def-a294-56d4da06251e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:49.900963 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c21e7-52ca-4def-a294-56d4da06251e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:49.901331 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c21e7-52ca-4def-a294-56d4da06251e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:49.901692 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c21e7-52ca-4def-a294-56d4da06251e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:50.004717 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c21e7-52ca-4def-a294-56d4da06251e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:50.006462 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c21e7-52ca-4def-a294-56d4da06251e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:50.091079 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:50.117925 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 216/428] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:54:50.127491 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e7d31-11ef-4ec3-9146-fac3bbfe6c01 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:50.247166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e7d31-11ef-4ec3-9146-fac3bbfe6c01 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:50.290984 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a288d7-93f5-43c3-972e-919ceff03c7d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:50.291330 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a288d7-93f5-43c3-972e-919ceff03c7d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:50.291609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a288d7-93f5-43c3-972e-919ceff03c7d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:50.293383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 217/429] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:50.293638 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4cece9-1d89-4efd-b531-32894b3a2b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:50.294533 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cece9-1d89-4efd-b531-32894b3a2b3a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:50.295011 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cece9-1d89-4efd-b531-32894b3a2b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:50.295546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cece9-1d89-4efd-b531-32894b3a2b3a None admin] PATH_INFO: `/v3/projects/1570ef44f3134332b1fc668345a37479` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:50.296956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4cece9-1d89-4efd-b531-32894b3a2b3a None admin] RBAC: Authorizing `identity:get_project(project_id=1570ef44f3134332b1fc668345a37479)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:50.298937 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4cece9-1d89-4efd-b531-32894b3a2b3a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:50.299335 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af88d5f-3b4b-409d-9c22-b01a76adb4f9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:50.299617 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af88d5f-3b4b-409d-9c22-b01a76adb4f9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:50.299889 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af88d5f-3b4b-409d-9c22-b01a76adb4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:50.301698 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 213/430] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:50 2022] GET /identity/v3/projects/1570ef44f3134332b1fc668345a37479 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:50.307482 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4cece9-1d89-4efd-b531-32894b3a2b3a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:50.323081 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0af88d5f-3b4b-409d-9c22-b01a76adb4f9 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:50.324956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0af88d5f-3b4b-409d-9c22-b01a76adb4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:50.374667 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:50.415540 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 218/431] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:54:50.416839 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02222ab-71c8-441e-b22e-f9dc738489cf None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:50.417213 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02222ab-71c8-441e-b22e-f9dc738489cf None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:50.417393 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02222ab-71c8-441e-b22e-f9dc738489cf None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:50.419310 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 219/432] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:50.425696 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d14a49-b778-44f6-bac4-ed588b2fc45b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:50.425985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d14a49-b778-44f6-bac4-ed588b2fc45b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:50.426268 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d14a49-b778-44f6-bac4-ed588b2fc45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:50.433105 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4cece9-1d89-4efd-b531-32894b3a2b3a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:50.451347 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b0d14a49-b778-44f6-bac4-ed588b2fc45b None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:50.452582 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b0d14a49-b778-44f6-bac4-ed588b2fc45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:50.471698 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea495c6-3817-4acc-92bf-c7facdbf0eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:50.472323 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea495c6-3817-4acc-92bf-c7facdbf0eda None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:50.472598 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea495c6-3817-4acc-92bf-c7facdbf0eda None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:50.472867 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea495c6-3817-4acc-92bf-c7facdbf0eda None admin] PATH_INFO: `/v3/groups/0271954fa90a4c699c1864243d5b4040` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:50.474100 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea495c6-3817-4acc-92bf-c7facdbf0eda None admin] RBAC: Authorizing `identity:get_group(group_id=0271954fa90a4c699c1864243d5b4040)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:50.475629 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea495c6-3817-4acc-92bf-c7facdbf0eda None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:50.477500 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 214/433] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 10 16:54:50 2022] GET /identity/v3/groups/0271954fa90a4c699c1864243d5b4040 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:50.478291 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea495c6-3817-4acc-92bf-c7facdbf0eda None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:50.531267 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 220/434] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:50.532125 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d14a49-b778-44f6-bac4-ed588b2fc45b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:50.576333 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:50.596178 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea495c6-3817-4acc-92bf-c7facdbf0eda None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:50.617841 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:50.636340 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb989a4-b8d2-42fa-98b0-8b2e0a100444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:50.637419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb989a4-b8d2-42fa-98b0-8b2e0a100444 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:50.637691 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb989a4-b8d2-42fa-98b0-8b2e0a100444 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:50.637962 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb989a4-b8d2-42fa-98b0-8b2e0a100444 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:50.649698 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb989a4-b8d2-42fa-98b0-8b2e0a100444 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:50.654467 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d14a49-b778-44f6-bac4-ed588b2fc45b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:50.655558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-bdb989a4-b8d2-42fa-98b0-8b2e0a100444 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:54:50.655558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:50.655558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:50.655558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:50.655558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:50.655558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:50.655558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:50.655558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:50.655558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:50.655558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:50.655558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:50.655558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:50.655558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:50.655558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:50.655558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:50.655558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:50.656549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:50.656549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:50.656549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:50.656549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:50.656549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:50.656549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:50.656549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:50.656549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:50.656549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:50.656549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:50.656549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:50.656549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:50.656549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:50.656549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:50.656549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:50.656549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:50.657419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:50.657419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:50.657419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:50.657419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:50.657419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:50.657419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:50.657419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:50.657419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:50.657419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:50.657419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:50.657419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:54:50.657419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:50.660031 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 215/435] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 10 16:54:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:50.660751 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb989a4-b8d2-42fa-98b0-8b2e0a100444 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:50.696141 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c591b-dbb4-4c57-a4bc-ba8e36281d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:50.696791 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c591b-dbb4-4c57-a4bc-ba8e36281d09 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:50.697080 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c591b-dbb4-4c57-a4bc-ba8e36281d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:50.697349 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c591b-dbb4-4c57-a4bc-ba8e36281d09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:50.698075 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c591b-dbb4-4c57-a4bc-ba8e36281d09 None admin] RBAC: Adding query filter params (group.id=0271954fa90a4c699c1864243d5b4040, role.id=a3e5816caacc425c9a6dd381fe0eddf0, scope.project.id=1570ef44f3134332b1fc668345a37479) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:50.698400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c591b-dbb4-4c57-a4bc-ba8e36281d09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:50.699868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c591b-dbb4-4c57-a4bc-ba8e36281d09 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:50.705906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 221/436] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 10 16:54:50 2022] GET /identity/v3/role_assignments?group.id=0271954fa90a4c699c1864243d5b4040&scope.project.id=1570ef44f3134332b1fc668345a37479&role.id=a3e5816caacc425c9a6dd381fe0eddf0 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:50.707273 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c591b-dbb4-4c57-a4bc-ba8e36281d09 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:50.775483 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb989a4-b8d2-42fa-98b0-8b2e0a100444 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:50.812895 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b226b-7825-44c2-984b-ef31745721f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:50.813537 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b226b-7825-44c2-984b-ef31745721f0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:50.813817 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b226b-7825-44c2-984b-ef31745721f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:50.814092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b226b-7825-44c2-984b-ef31745721f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:50.827215 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b226b-7825-44c2-984b-ef31745721f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:50.828384 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:50.832579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-4c8b226b-7825-44c2-984b-ef31745721f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:50.832579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:50.832579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:50.832579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:50.832579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:50.832579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:50.832579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:50.832579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:50.832579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:50.832579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:50.832579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:50.832579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:50.832579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:50.832579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:50.832579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:50.832579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:50.833535 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:50.833535 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:50.833535 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:50.833535 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:50.833535 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:50.833535 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:50.833535 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:50.833535 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:50.833535 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:50.833535 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:50.833535 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:50.833535 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:50.833535 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:50.833535 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:50.833535 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:50.833535 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:50.834393 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:50.834393 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:50.834393 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:50.834393 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:50.834393 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:50.834393 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:50.834393 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:50.834393 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:50.834393 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:50.834393 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:50.834393 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 16:54:50.834393 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:50.834393 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 216/437] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:50.836451 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c591b-dbb4-4c57-a4bc-ba8e36281d09 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:50.840621 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b226b-7825-44c2-984b-ef31745721f0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:50.877743 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10398cd3-8af5-4648-b0fb-c87325f12f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:50.878743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:50.879943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10398cd3-8af5-4648-b0fb-c87325f12f79 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:50.880419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10398cd3-8af5-4648-b0fb-c87325f12f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:50.880877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10398cd3-8af5-4648-b0fb-c87325f12f79 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:50.881761 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10398cd3-8af5-4648-b0fb-c87325f12f79 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:50.882277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10398cd3-8af5-4648-b0fb-c87325f12f79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:50.883680 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10398cd3-8af5-4648-b0fb-c87325f12f79 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:50.895337 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 222/438] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:54:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:50.898749 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10398cd3-8af5-4648-b0fb-c87325f12f79 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:50.958342 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b226b-7825-44c2-984b-ef31745721f0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:51.011419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814048a4-d29d-4082-a331-67843985a798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:51.012617 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814048a4-d29d-4082-a331-67843985a798 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:51.013062 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814048a4-d29d-4082-a331-67843985a798 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:51.013524 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814048a4-d29d-4082-a331-67843985a798 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:51.014248 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814048a4-d29d-4082-a331-67843985a798 None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:51.014611 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814048a4-d29d-4082-a331-67843985a798 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:51.016360 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814048a4-d29d-4082-a331-67843985a798 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:51.023803 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10398cd3-8af5-4648-b0fb-c87325f12f79 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:51.029686 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 217/439] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:54:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:51.033550 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814048a4-d29d-4082-a331-67843985a798 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:51.070250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da303896-7a6d-4e93-8716-ca1bbeb54a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:51.070998 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da303896-7a6d-4e93-8716-ca1bbeb54a82 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:51.071276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da303896-7a6d-4e93-8716-ca1bbeb54a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:51.071543 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da303896-7a6d-4e93-8716-ca1bbeb54a82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:51.108930 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:51.155013 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da303896-7a6d-4e93-8716-ca1bbeb54a82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:51.156961 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da303896-7a6d-4e93-8716-ca1bbeb54a82 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:51.159584 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:51.163527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814048a4-d29d-4082-a331-67843985a798 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:51.212923 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14a845-c3d3-466a-af5b-ea8d85d79b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:51.213523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14a845-c3d3-466a-af5b-ea8d85d79b15 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:51.213800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14a845-c3d3-466a-af5b-ea8d85d79b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:51.214068 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14a845-c3d3-466a-af5b-ea8d85d79b15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:51.272702 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 223/440] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 16:54:51.278918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da303896-7a6d-4e93-8716-ca1bbeb54a82 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:51.285716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14a845-c3d3-466a-af5b-ea8d85d79b15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:51.287218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14a845-c3d3-466a-af5b-ea8d85d79b15 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:51.336762 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:51.377305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da303896-7a6d-4e93-8716-ca1bbeb54a82 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:51.382588 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:51.387690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 218/441] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:54:51.389035 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5759828b-2c98-4639-9421-cb137c5df739 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:51.389258 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5759828b-2c98-4639-9421-cb137c5df739 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:51.389527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5759828b-2c98-4639-9421-cb137c5df739 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:51.390949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 219/442] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:51.393077 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5759828b-2c98-4639-9421-cb137c5df739 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:51.419948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b794b2-bc82-4e78-a7e4-1d4cbf2429ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:51.420797 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b794b2-bc82-4e78-a7e4-1d4cbf2429ba None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:51.420797 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b794b2-bc82-4e78-a7e4-1d4cbf2429ba None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:51.421072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b794b2-bc82-4e78-a7e4-1d4cbf2429ba None admin] PATH_INFO: `/v3/projects/793400b224a54914900adad496869b83` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:51.422237 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b794b2-bc82-4e78-a7e4-1d4cbf2429ba None admin] RBAC: Authorizing `identity:get_project(project_id=793400b224a54914900adad496869b83)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:51.423699 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b794b2-bc82-4e78-a7e4-1d4cbf2429ba None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:51.425447 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 224/443] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:51 2022] GET /identity/v3/projects/793400b224a54914900adad496869b83 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:51.427457 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620d244-0f82-443e-8f02-143e62fdac5f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:51.427457 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620d244-0f82-443e-8f02-143e62fdac5f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:51.427457 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620d244-0f82-443e-8f02-143e62fdac5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:51.444991 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b620d244-0f82-443e-8f02-143e62fdac5f None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:51.446482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b620d244-0f82-443e-8f02-143e62fdac5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:51.505196 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5759828b-2c98-4639-9421-cb137c5df739 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:51.513756 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 225/444] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:51.514685 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b620d244-0f82-443e-8f02-143e62fdac5f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:51.542023 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53428d0-ebc6-4cdd-9971-10c96a913385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:51.542722 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53428d0-ebc6-4cdd-9971-10c96a913385 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:51.542981 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53428d0-ebc6-4cdd-9971-10c96a913385 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:51.543244 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53428d0-ebc6-4cdd-9971-10c96a913385 None admin] PATH_INFO: `/v3/projects/793400b224a54914900adad496869b83` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:51.544352 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53428d0-ebc6-4cdd-9971-10c96a913385 None admin] RBAC: Authorizing `identity:get_project(project_id=793400b224a54914900adad496869b83)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:51.545784 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53428d0-ebc6-4cdd-9971-10c96a913385 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:51.547741 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 220/445] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:51 2022] GET /identity/v3/projects/793400b224a54914900adad496869b83 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:51.548483 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53428d0-ebc6-4cdd-9971-10c96a913385 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:51.573602 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:51.615469 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b620d244-0f82-443e-8f02-143e62fdac5f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:51.617617 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:51.651212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad5010-f2c1-468e-8cc5-50c3e38721e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:51.651839 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad5010-f2c1-468e-8cc5-50c3e38721e3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:51.652214 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad5010-f2c1-468e-8cc5-50c3e38721e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:51.652487 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad5010-f2c1-468e-8cc5-50c3e38721e3 None admin] PATH_INFO: `/v3/groups/0271954fa90a4c699c1864243d5b4040` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:51.653583 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad5010-f2c1-468e-8cc5-50c3e38721e3 None admin] RBAC: Authorizing `identity:get_group(group_id=0271954fa90a4c699c1864243d5b4040)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:51.654814 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53428d0-ebc6-4cdd-9971-10c96a913385 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:51.655050 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad5010-f2c1-468e-8cc5-50c3e38721e3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:51.656602 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 226/446] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 10 16:54:51 2022] GET /identity/v3/groups/0271954fa90a4c699c1864243d5b4040 => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:51.661598 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad5010-f2c1-468e-8cc5-50c3e38721e3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:51.688806 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536d8ee-97f2-495d-b358-b705029537ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:51.689359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536d8ee-97f2-495d-b358-b705029537ff None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:51.689634 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536d8ee-97f2-495d-b358-b705029537ff None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:51.689904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536d8ee-97f2-495d-b358-b705029537ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:51.700504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536d8ee-97f2-495d-b358-b705029537ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:51.705237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-6536d8ee-97f2-495d-b358-b705029537ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:54:51.705237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:51.705237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:51.705237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:51.705237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:51.705237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:51.705237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:51.705237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:51.705237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:51.705237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:51.705237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:51.705237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:51.705237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:51.705237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:51.705237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:51.705237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:51.706523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:51.706523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:51.706523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:51.706523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:51.706523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:51.706523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:51.706523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:51.706523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:51.706523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:51.706523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:51.706523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:51.706523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:51.706523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:51.706523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:51.706523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:51.706523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:51.707538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:51.707538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:51.707538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:51.707538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:51.707538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:51.707538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:51.707538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:51.707538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:51.707538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:51.707538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:51.707538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:54:51.707538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:51.707538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 221/447] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:54:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:51.707538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536d8ee-97f2-495d-b358-b705029537ff None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:51.760296 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad5010-f2c1-468e-8cc5-50c3e38721e3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:51.785791 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:51.794670 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c0f6e-44d0-4a48-9929-f2fd230c1bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:51.795298 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c0f6e-44d0-4a48-9929-f2fd230c1bdd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:51.795569 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c0f6e-44d0-4a48-9929-f2fd230c1bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:51.795839 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c0f6e-44d0-4a48-9929-f2fd230c1bdd None admin] PATH_INFO: `/v3/groups/0271954fa90a4c699c1864243d5b4040` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:51.796937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c0f6e-44d0-4a48-9929-f2fd230c1bdd None admin] RBAC: Authorizing `identity:get_group(group_id=0271954fa90a4c699c1864243d5b4040)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:51.800939 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c0f6e-44d0-4a48-9929-f2fd230c1bdd None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:51.800939 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 227/448] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 10 16:54:51 2022] GET /identity/v3/groups/0271954fa90a4c699c1864243d5b4040 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:51.800939 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c0f6e-44d0-4a48-9929-f2fd230c1bdd None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:51.812013 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536d8ee-97f2-495d-b358-b705029537ff None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:51.838187 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:51.846618 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53224357-5e4d-4f4c-aeab-249f314710c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:51.847218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53224357-5e4d-4f4c-aeab-249f314710c6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:51.847493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53224357-5e4d-4f4c-aeab-249f314710c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:51.847772 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53224357-5e4d-4f4c-aeab-249f314710c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:51.852043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53224357-5e4d-4f4c-aeab-249f314710c6 None admin] RBAC: Adding query filter params (group.id=0271954fa90a4c699c1864243d5b4040, role.id=a3e5816caacc425c9a6dd381fe0eddf0, scope.project.id=793400b224a54914900adad496869b83) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:51.852043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53224357-5e4d-4f4c-aeab-249f314710c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:51.852043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53224357-5e4d-4f4c-aeab-249f314710c6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:51.857038 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 222/449] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 10 16:54:51 2022] GET /identity/v3/role_assignments?group.id=0271954fa90a4c699c1864243d5b4040&scope.project.id=793400b224a54914900adad496869b83&role.id=a3e5816caacc425c9a6dd381fe0eddf0 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:51.857819 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53224357-5e4d-4f4c-aeab-249f314710c6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:51.916112 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c0f6e-44d0-4a48-9929-f2fd230c1bdd None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:51.961491 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cea947-be9d-4982-ab84-aa73bd553aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:51.962027 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cea947-be9d-4982-ab84-aa73bd553aa0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:51.962305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cea947-be9d-4982-ab84-aa73bd553aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:51.962611 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cea947-be9d-4982-ab84-aa73bd553aa0 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:51.963284 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cea947-be9d-4982-ab84-aa73bd553aa0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:51.963583 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cea947-be9d-4982-ab84-aa73bd553aa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:51.965160 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cea947-be9d-4982-ab84-aa73bd553aa0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:51.974445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 228/450] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:54:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:51.975420 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53224357-5e4d-4f4c-aeab-249f314710c6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:51.980991 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cea947-be9d-4982-ab84-aa73bd553aa0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:52.016584 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a7252-c394-46cc-910e-a8790a12fc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:52.017496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a7252-c394-46cc-910e-a8790a12fc99 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:52.017775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a7252-c394-46cc-910e-a8790a12fc99 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:52.018051 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a7252-c394-46cc-910e-a8790a12fc99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:52.018822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a7252-c394-46cc-910e-a8790a12fc99 None admin] RBAC: Adding query filter params (group.id=0271954fa90a4c699c1864243d5b4040, role.id=dfb72dc1f3744c8e957abcc4ac967dac, scope.project.id=793400b224a54914900adad496869b83) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:52.019133 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a7252-c394-46cc-910e-a8790a12fc99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:52.020570 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a7252-c394-46cc-910e-a8790a12fc99 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:52.026832 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 223/451] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 10 16:54:51 2022] GET /identity/v3/role_assignments?group.id=0271954fa90a4c699c1864243d5b4040&scope.project.id=793400b224a54914900adad496869b83&role.id=dfb72dc1f3744c8e957abcc4ac967dac => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:52.049088 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:52.101756 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cea947-be9d-4982-ab84-aa73bd553aa0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:52.140770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f75a17-0ff8-43cd-b201-bba29a44d2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:52.141394 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f75a17-0ff8-43cd-b201-bba29a44d2d6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:52.141670 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f75a17-0ff8-43cd-b201-bba29a44d2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:52.141938 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f75a17-0ff8-43cd-b201-bba29a44d2d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:52.213937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f75a17-0ff8-43cd-b201-bba29a44d2d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:52.215965 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f75a17-0ff8-43cd-b201-bba29a44d2d6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:52.271633 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:52.319411 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 229/452] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:54:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 16:54:52.326126 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a7252-c394-46cc-910e-a8790a12fc99 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:52.382840 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:52.429055 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a7252-c394-46cc-910e-a8790a12fc99 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:52.467082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4c24e-aba8-4e2f-b0ac-c48048a4a4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:52.467637 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4c24e-aba8-4e2f-b0ac-c48048a4a4bc None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:52.467913 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4c24e-aba8-4e2f-b0ac-c48048a4a4bc None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:52.468184 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4c24e-aba8-4e2f-b0ac-c48048a4a4bc None admin] PATH_INFO: `/v3/projects/aff7d0faeed34f6193c691567931a8ff` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:52.469351 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4c24e-aba8-4e2f-b0ac-c48048a4a4bc None admin] RBAC: Authorizing `identity:get_project(project_id=aff7d0faeed34f6193c691567931a8ff)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:52.470852 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4c24e-aba8-4e2f-b0ac-c48048a4a4bc None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:52.472529 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 224/453] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:54:52 2022] GET /identity/v3/projects/aff7d0faeed34f6193c691567931a8ff => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:52.478396 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f75a17-0ff8-43cd-b201-bba29a44d2d6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:52.577499 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f75a17-0ff8-43cd-b201-bba29a44d2d6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:52.610885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49c0d6-57eb-44b1-8196-9c6f57a02e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:52.611514 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49c0d6-57eb-44b1-8196-9c6f57a02e51 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:52.611800 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49c0d6-57eb-44b1-8196-9c6f57a02e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:52.612065 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49c0d6-57eb-44b1-8196-9c6f57a02e51 None admin] PATH_INFO: `/v3/groups/3d6df3ac551648d18ab59edbc7a51394` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:52.613170 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49c0d6-57eb-44b1-8196-9c6f57a02e51 None admin] RBAC: Authorizing `identity:get_group(group_id=3d6df3ac551648d18ab59edbc7a51394)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:52.614642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49c0d6-57eb-44b1-8196-9c6f57a02e51 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:52.616209 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 230/454] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 10 16:54:52 2022] GET /identity/v3/groups/3d6df3ac551648d18ab59edbc7a51394 => generated 229 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:52.622174 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4c24e-aba8-4e2f-b0ac-c48048a4a4bc None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:52.722541 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4c24e-aba8-4e2f-b0ac-c48048a4a4bc None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:52.744108 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:52.755234 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a31ae5-c562-4403-9b46-b82caab03a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:52.755823 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a31ae5-c562-4403-9b46-b82caab03a0c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:52.756100 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a31ae5-c562-4403-9b46-b82caab03a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:52.758028 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a31ae5-c562-4403-9b46-b82caab03a0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:52.758028 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a31ae5-c562-4403-9b46-b82caab03a0c None admin] RBAC: Adding query filter params (group.id=3d6df3ac551648d18ab59edbc7a51394, role.id=7554b353873942fab506de830188aba0, scope.project.id=aff7d0faeed34f6193c691567931a8ff) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:52.758028 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a31ae5-c562-4403-9b46-b82caab03a0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:52.758961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a31ae5-c562-4403-9b46-b82caab03a0c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:52.765315 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 225/455] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 10 16:54:52 2022] GET /identity/v3/role_assignments?group.id=3d6df3ac551648d18ab59edbc7a51394&scope.project.id=aff7d0faeed34f6193c691567931a8ff&role.id=7554b353873942fab506de830188aba0 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:55.404751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63f81f-d435-4193-a745-72ea326689ec None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:55.405265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63f81f-d435-4193-a745-72ea326689ec None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:55.405265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63f81f-d435-4193-a745-72ea326689ec None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:55.406945 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 231/456] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:55.413993 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5cf27-384d-480c-a5e0-42669d2c10e1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:55.414892 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5cf27-384d-480c-a5e0-42669d2c10e1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:55.415362 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5cf27-384d-480c-a5e0-42669d2c10e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:55.439436 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a8b5cf27-384d-480c-a5e0-42669d2c10e1 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:55.448482 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a8b5cf27-384d-480c-a5e0-42669d2c10e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:55.544047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 226/457] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:55.616840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63f81f-d435-4193-a745-72ea326689ec None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:55.681001 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:55.832587 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63f81f-d435-4193-a745-72ea326689ec None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:55.880385 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9751621-a379-4439-b8fa-08541c81527d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:55.881015 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9751621-a379-4439-b8fa-08541c81527d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:55.881293 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9751621-a379-4439-b8fa-08541c81527d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:55.881563 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9751621-a379-4439-b8fa-08541c81527d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:55.898329 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9751621-a379-4439-b8fa-08541c81527d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:55.900098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9751621-a379-4439-b8fa-08541c81527d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:55.917138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-b9751621-a379-4439-b8fa-08541c81527d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:54:55.917138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:55.917138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:55.917138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:55.917138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:55.917138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:55.917138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:55.917138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:55.917138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:55.917138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:55.917138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:55.917138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:55.917138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:54:55.917138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:54:55.917138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:54:55.917138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:54:55.918496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:55.918496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:55.918496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:55.918496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:55.918496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:55.918496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:55.918496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:55.918496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:55.918496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:55.918496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:55.918496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:55.918496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:55.918496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:55.918496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:55.918496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:55.918496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:55.918496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:54:55.920222 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:54:55.920222 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:54:55.920222 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:54:55.920222 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 232/458] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:54:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:54:55.937455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5cf27-384d-480c-a5e0-42669d2c10e1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:56.136110 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5cf27-384d-480c-a5e0-42669d2c10e1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:56.155719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:56.189885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ae12f-6aa5-4bb8-83d0-4cbf97e747aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:56.190539 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ae12f-6aa5-4bb8-83d0-4cbf97e747aa None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:56.190814 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ae12f-6aa5-4bb8-83d0-4cbf97e747aa None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:56.191078 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ae12f-6aa5-4bb8-83d0-4cbf97e747aa None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:56.191850 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582ae12f-6aa5-4bb8-83d0-4cbf97e747aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:56.192052 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582ae12f-6aa5-4bb8-83d0-4cbf97e747aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:56.193388 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582ae12f-6aa5-4bb8-83d0-4cbf97e747aa None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:56.219104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 227/459] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:54:56.225488 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9751621-a379-4439-b8fa-08541c81527d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:56.368773 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:56.400726 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9751621-a379-4439-b8fa-08541c81527d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:56.455821 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83822627-7ac1-4aa9-a9b4-4b0a07747e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:56.456422 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83822627-7ac1-4aa9-a9b4-4b0a07747e0e None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:56.456675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83822627-7ac1-4aa9-a9b4-4b0a07747e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:56.456956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83822627-7ac1-4aa9-a9b4-4b0a07747e0e None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:56.457660 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83822627-7ac1-4aa9-a9b4-4b0a07747e0e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:56.459156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83822627-7ac1-4aa9-a9b4-4b0a07747e0e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:56.525568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3256f-24c1-4c99-b1df-a14b5dfafc44 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:56.527516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3256f-24c1-4c99-b1df-a14b5dfafc44 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:56.528175 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3256f-24c1-4c99-b1df-a14b5dfafc44 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:56.531191 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 228/460] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:56.536853 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98725700-9c95-47a7-a698-a298255e54a4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:56.537309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98725700-9c95-47a7-a698-a298255e54a4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:56.537719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98725700-9c95-47a7-a698-a298255e54a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:56.559814 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-98725700-9c95-47a7-a698-a298255e54a4 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:56.561250 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-98725700-9c95-47a7-a698-a298255e54a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:56.594599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 233/461] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 10 16:54:56 2022] POST /identity/v3/users => generated 246 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:56.673426 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b769e-6c4b-45d9-a9a5-076e55068c27 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:56.673768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b769e-6c4b-45d9-a9a5-076e55068c27 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:56.673984 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b769e-6c4b-45d9-a9a5-076e55068c27 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:56.676474 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 234/462] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:56.687682 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6b4cf-8853-41aa-ab8a-44a2ee9ec2aa None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:56.687995 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6b4cf-8853-41aa-ab8a-44a2ee9ec2aa None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:56.694380 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6b4cf-8853-41aa-ab8a-44a2ee9ec2aa None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:56.694591 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 229/464] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:54:56.694688 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 235/464] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:56.694759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925f575-3d7d-4750-9a89-87d609e0832c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:56.694759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925f575-3d7d-4750-9a89-87d609e0832c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:56.694759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925f575-3d7d-4750-9a89-87d609e0832c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:56.706004 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa4bf8-0b8d-4924-963a-5831476bd1d1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:56.706269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa4bf8-0b8d-4924-963a-5831476bd1d1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:56.706505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa4bf8-0b8d-4924-963a-5831476bd1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:56.730845 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3925f575-3d7d-4750-9a89-87d609e0832c None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:56.730845 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3925f575-3d7d-4750-9a89-87d609e0832c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:56.735877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c0fa4bf8-0b8d-4924-963a-5831476bd1d1 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:56.737097 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c0fa4bf8-0b8d-4924-963a-5831476bd1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:56.842301 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 230/465] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:56.843066 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925f575-3d7d-4750-9a89-87d609e0832c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:56.851098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 236/466] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:56.871959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa4bf8-0b8d-4924-963a-5831476bd1d1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:56.892310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:56.965735 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:57.008092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925f575-3d7d-4750-9a89-87d609e0832c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:57.020957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa4bf8-0b8d-4924-963a-5831476bd1d1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:57.046326 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa323b-daad-459a-a6e4-79ee92be9e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:57.047122 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa323b-daad-459a-a6e4-79ee92be9e24 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:57.047496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa323b-daad-459a-a6e4-79ee92be9e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:57.047907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa323b-daad-459a-a6e4-79ee92be9e24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:57.059648 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4aac40-d02a-442b-a81a-2881c895d2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:57.060543 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4aac40-d02a-442b-a81a-2881c895d2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:57.060965 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4aac40-d02a-442b-a81a-2881c895d2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:57.061387 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4aac40-d02a-442b-a81a-2881c895d2b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:57.061777 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aa323b-daad-459a-a6e4-79ee92be9e24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:57.063367 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aa323b-daad-459a-a6e4-79ee92be9e24 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:57.073653 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4aac40-d02a-442b-a81a-2881c895d2b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:57.075413 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4aac40-d02a-442b-a81a-2881c895d2b0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:57.080400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-a1aa323b-daad-459a-a6e4-79ee92be9e24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:54:57.080400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:57.080400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:57.080400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:57.080400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:57.080400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:57.080400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:57.080400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:57.080400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:57.080400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:57.080400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:57.080400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:57.080400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:54:57.080400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:54:57.080400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:54:57.080400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:54:57.081455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:57.081455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:57.081455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:57.081455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:57.081455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:57.081455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:57.081455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:57.081455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:57.081455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:57.081455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:57.081455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:57.081455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:57.081455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:57.081455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:57.081455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:57.081455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:57.081455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:54:57.082335 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:54:57.082335 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:54:57.082335 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:57.082335 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 231/467] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:54:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:57.083322 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa323b-daad-459a-a6e4-79ee92be9e24 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:57.096806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ef4aac40-d02a-442b-a81a-2881c895d2b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:54:57.096806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:57.096806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:57.096806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:57.096806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:57.096806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:57.096806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:57.096806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:57.096806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:57.096806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:57.096806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:57.096806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:57.096806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:54:57.096806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:54:57.096806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:54:57.096806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:54:57.097774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:57.097774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:57.097774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:57.097774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:57.097774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:57.097774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:57.097774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:57.097774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:57.097774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:57.097774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:57.097774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:57.097774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:57.097774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:57.097774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:57.097774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:57.097774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:57.097774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:54:57.099177 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:54:57.099177 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:54:57.099177 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:54:57.102801 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 237/468] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:54:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:57.103784 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80009c-7ad8-478d-bf80-a5c1d911eb89 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:57.104139 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80009c-7ad8-478d-bf80-a5c1d911eb89 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:57.104375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80009c-7ad8-478d-bf80-a5c1d911eb89 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:57.106617 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 238/469] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:57.107038 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80009c-7ad8-478d-bf80-a5c1d911eb89 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:57.177606 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:57.190866 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa323b-daad-459a-a6e4-79ee92be9e24 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:57.220098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80009c-7ad8-478d-bf80-a5c1d911eb89 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:57.226434 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee762ac5-a0d3-48b9-94ee-e7bf6fd0712a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:57.227061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee762ac5-a0d3-48b9-94ee-e7bf6fd0712a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:57.227338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee762ac5-a0d3-48b9-94ee-e7bf6fd0712a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:57.227604 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee762ac5-a0d3-48b9-94ee-e7bf6fd0712a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:57.236512 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:57.239797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee762ac5-a0d3-48b9-94ee-e7bf6fd0712a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:57.241268 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee762ac5-a0d3-48b9-94ee-e7bf6fd0712a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:57.257033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-ee762ac5-a0d3-48b9-94ee-e7bf6fd0712a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:54:57.257033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:57.257033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:57.257033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:57.257033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:57.257033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:57.257033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:57.257033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:57.257033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:57.257033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:57.257033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:57.257033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:57.257033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:54:57.257033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:54:57.257033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:54:57.257033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:54:57.257984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:57.257984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:57.257984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:57.257984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:57.257984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:57.257984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:57.257984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:57.257984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:57.257984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:57.257984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:57.257984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:57.257984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:57.257984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:57.257984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:57.257984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:57.257984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:57.257984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:54:57.258945 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:54:57.258945 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:54:57.258945 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:57.258945 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 232/470] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:54:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:57.258945 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee762ac5-a0d3-48b9-94ee-e7bf6fd0712a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:57.264806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ecf38-2bab-4630-82d5-3f4744ec46d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:57.265664 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ecf38-2bab-4630-82d5-3f4744ec46d9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:57.266088 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ecf38-2bab-4630-82d5-3f4744ec46d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:57.266527 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ecf38-2bab-4630-82d5-3f4744ec46d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:57.267359 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593ecf38-2bab-4630-82d5-3f4744ec46d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:57.267667 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593ecf38-2bab-4630-82d5-3f4744ec46d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:57.269069 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593ecf38-2bab-4630-82d5-3f4744ec46d9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:57.286518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 239/471] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:54:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:57.286518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758f973-e1e5-4e5d-b1b7-14eaa4b823d7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:57.286518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758f973-e1e5-4e5d-b1b7-14eaa4b823d7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:57.286518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758f973-e1e5-4e5d-b1b7-14eaa4b823d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:57.302943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6758f973-e1e5-4e5d-b1b7-14eaa4b823d7 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:57.304130 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6758f973-e1e5-4e5d-b1b7-14eaa4b823d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:57.373595 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee762ac5-a0d3-48b9-94ee-e7bf6fd0712a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:57.378476 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 240/472] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:57.378476 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758f973-e1e5-4e5d-b1b7-14eaa4b823d7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:57.415639 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac187c8-2290-4a0a-a029-29789ab9cbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:57.416238 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac187c8-2290-4a0a-a029-29789ab9cbcd None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:57.416514 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac187c8-2290-4a0a-a029-29789ab9cbcd None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:57.416783 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac187c8-2290-4a0a-a029-29789ab9cbcd None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:57.417442 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac187c8-2290-4a0a-a029-29789ab9cbcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:57.417814 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac187c8-2290-4a0a-a029-29789ab9cbcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:57.419426 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac187c8-2290-4a0a-a029-29789ab9cbcd None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:57.428199 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:57.431650 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 233/473] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:54:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:57.432509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac187c8-2290-4a0a-a029-29789ab9cbcd None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:57.506728 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758f973-e1e5-4e5d-b1b7-14eaa4b823d7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:57.509684 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:57.539511 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac187c8-2290-4a0a-a029-29789ab9cbcd None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:57.542611 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0091902b-45fe-438d-9c97-f8102656e1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:57.547091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091902b-45fe-438d-9c97-f8102656e1ac None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:57.547091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091902b-45fe-438d-9c97-f8102656e1ac None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:57.547091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091902b-45fe-438d-9c97-f8102656e1ac None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:57.547091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0091902b-45fe-438d-9c97-f8102656e1ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:57.547091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0091902b-45fe-438d-9c97-f8102656e1ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:57.547091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0091902b-45fe-438d-9c97-f8102656e1ac None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:57.557272 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 241/474] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:54:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:57.557882 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0091902b-45fe-438d-9c97-f8102656e1ac None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:57.582789 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbfe452-dbc7-4fa0-bb33-4fa14925d072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:57.583428 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbfe452-dbc7-4fa0-bb33-4fa14925d072 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:57.583655 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbfe452-dbc7-4fa0-bb33-4fa14925d072 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:57.583922 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbfe452-dbc7-4fa0-bb33-4fa14925d072 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:57.584548 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbfe452-dbc7-4fa0-bb33-4fa14925d072 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:57.585978 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbfe452-dbc7-4fa0-bb33-4fa14925d072 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:57.660408 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0091902b-45fe-438d-9c97-f8102656e1ac None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:57.678402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 234/475] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 10 16:54:57 2022] POST /identity/v3/users => generated 249 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:57.678402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbfe452-dbc7-4fa0-bb33-4fa14925d072 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:57.706254 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e222cc0-c6f0-468f-a06d-7eeb6845ec91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:57.706906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e222cc0-c6f0-468f-a06d-7eeb6845ec91 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:57.707184 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e222cc0-c6f0-468f-a06d-7eeb6845ec91 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:57.707454 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e222cc0-c6f0-468f-a06d-7eeb6845ec91 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:57.708128 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e222cc0-c6f0-468f-a06d-7eeb6845ec91 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:57.709775 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e222cc0-c6f0-468f-a06d-7eeb6845ec91 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:57.740512 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:57.782484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.sql.core [None req-2e222cc0-c6f0-468f-a06d-7eeb6845ec91 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 16:54:57.782484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 10 16:54:57.782484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [parameters: {'id': 'e0a434604d9543a4901e597918308d33', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 10 16:54:57.782484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81611) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 16:54:57.790706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbfe452-dbc7-4fa0-bb33-4fa14925d072 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:57.800507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2e222cc0-c6f0-468f-a06d-7eeb6845ec91 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 10 16:54:57.800507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:57.800507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 16:54:57.800507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 16:54:57.800507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 16:54:57.800507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 16:54:57.800507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 16:54:57.800507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 16:54:57.800507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 16:54:57.800507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application conn.query(q) Aug 10 16:54:57.800507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 16:54:57.800507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 16:54:57.800507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 16:54:57.800507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result.read() Aug 10 16:54:57.800507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 16:54:57.801750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 16:54:57.801750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 16:54:57.801750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 16:54:57.801750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 16:54:57.801750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 16:54:57.801750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 16:54:57.801750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 16:54:57.801750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 16:54:57.801750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:54:57.801750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 16:54:57.801750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:54:57.801750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:57.801750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 16:54:57.801750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 16:54:57.801750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 10 16:54:57.801750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return ref.to_dict() Aug 10 16:54:57.801750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 16:54:57.801750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application next(self.gen) Aug 10 16:54:57.802906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 16:54:57.802906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application yield resource Aug 10 16:54:57.802906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 16:54:57.802906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application next(self.gen) Aug 10 16:54:57.802906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 16:54:57.802906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 16:54:57.802906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 16:54:57.802906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 16:54:57.802906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 16:54:57.802906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise self.value Aug 10 16:54:57.802906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 16:54:57.802906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 16:54:57.802906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 16:54:57.802906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application session.commit() Aug 10 16:54:57.802906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 16:54:57.802906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 16:54:57.802906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 16:54:57.803972 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 16:54:57.803972 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 16:54:57.803972 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.session.flush() Aug 10 16:54:57.803972 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 16:54:57.803972 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 16:54:57.803972 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 16:54:57.803972 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 16:54:57.803972 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 16:54:57.803972 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application compat.raise_( Aug 10 16:54:57.803972 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 16:54:57.803972 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception Aug 10 16:54:57.803972 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 16:54:57.803972 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 16:54:57.803972 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 16:54:57.803972 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 16:54:57.803972 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 16:54:57.803972 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 16:54:57.805067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 16:54:57.805067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 16:54:57.805067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 10 16:54:57.805067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 10 16:54:57.805067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 16:54:57.805067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 16:54:57.805067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 16:54:57.805067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 16:54:57.805067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 16:54:57.805067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 16:54:57.805067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 16:54:57.805067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 16:54:57.805067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 16:54:57.805067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 16:54:57.805067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 16:54:57.805067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception Aug 10 16:54:57.806285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 16:54:57.806285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 16:54:57.806285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 16:54:57.806285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 16:54:57.806285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 16:54:57.806285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 16:54:57.806285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 16:54:57.806285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application conn.query(q) Aug 10 16:54:57.806285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 16:54:57.806285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 16:54:57.806285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 16:54:57.806285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result.read() Aug 10 16:54:57.806285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 16:54:57.806285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 16:54:57.806285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 16:54:57.806285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 16:54:57.807400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 16:54:57.807400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 16:54:57.807400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 16:54:57.807400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 16:54:57.807400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 16:54:57.807400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 10 16:54:57.807400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [parameters: {'id': 'e0a434604d9543a4901e597918308d33', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 10 16:54:57.807400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 16:54:57.807400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:54:57.807400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:54:57.807400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:54:57.807400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:57.807400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:57.807400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:57.807400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:57.807400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:57.808988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:57.808988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:57.808988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:57.808988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:57.808988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:57.808988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:57.808988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 10 16:54:57.808988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 10 16:54:57.808988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:57.808988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:57.808988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 10 16:54:57.808988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 10 16:54:57.808988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 16:54:57.808988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 16:54:57.808988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 10 16:54:57.808988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:54:57.814555 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 242/476] 10.222.0.45 () {60 vars in 1131 bytes} [Wed Aug 10 16:54:57 2022] POST /identity/v3/roles => generated 144 bytes in 257 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:57.818569 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e222cc0-c6f0-468f-a06d-7eeb6845ec91 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:57.842993 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f3ece-8ae0-4b26-aed6-8517a0e4fd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:57.843718 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f3ece-8ae0-4b26-aed6-8517a0e4fd6f None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:57.844059 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f3ece-8ae0-4b26-aed6-8517a0e4fd6f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:57.844401 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f3ece-8ae0-4b26-aed6-8517a0e4fd6f None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:57.845359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f3ece-8ae0-4b26-aed6-8517a0e4fd6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:57.846913 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f3ece-8ae0-4b26-aed6-8517a0e4fd6f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:57.859172 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:57.940402 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e222cc0-c6f0-468f-a06d-7eeb6845ec91 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:57.974684 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af465175-9db1-48d9-8b12-7d3abe785966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:57.975294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af465175-9db1-48d9-8b12-7d3abe785966 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:57.975572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af465175-9db1-48d9-8b12-7d3abe785966 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:57.975856 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af465175-9db1-48d9-8b12-7d3abe785966 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:57.976506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af465175-9db1-48d9-8b12-7d3abe785966 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:57.977928 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af465175-9db1-48d9-8b12-7d3abe785966 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:58.051994 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 235/477] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 10 16:54:57 2022] POST /identity/v3/users => generated 248 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:58.053190 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f3ece-8ae0-4b26-aed6-8517a0e4fd6f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:58.086390 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 243/478] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 10 16:54:57 2022] POST /identity/v3/users => generated 248 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:54:58.256443 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f3ece-8ae0-4b26-aed6-8517a0e4fd6f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:58.282733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:58.298086 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9067b-3f5c-4baf-963e-32794ba5d0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:58.298830 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9067b-3f5c-4baf-963e-32794ba5d0b1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:58.299110 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9067b-3f5c-4baf-963e-32794ba5d0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:58.299377 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9067b-3f5c-4baf-963e-32794ba5d0b1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:58.312345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f9067b-3f5c-4baf-963e-32794ba5d0b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:58.317065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-70f9067b-3f5c-4baf-963e-32794ba5d0b1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:54:58.317065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:58.317065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:58.317065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:58.317065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:58.317065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:58.317065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:58.317065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:58.317065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:58.317065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:58.317065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:58.317065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:58.317065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:58.317065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:58.317065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:58.317065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:58.317991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:58.317991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:58.317991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:58.317991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:58.317991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:58.317991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:58.317991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:58.317991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:58.317991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:58.317991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:58.317991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:58.317991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:58.317991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:58.317991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:58.317991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:58.317991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:58.318848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:58.318848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:58.318848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:58.318848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:58.318848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:58.318848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:58.318848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:58.318848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:58.318848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:58.318848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:58.318848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 16:54:58.318848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:58.318848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 236/479] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 10 16:54:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:58.324287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af465175-9db1-48d9-8b12-7d3abe785966 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:58.425739 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:58.442993 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af465175-9db1-48d9-8b12-7d3abe785966 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:58.482052 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6f778-5731-434d-a76c-b9700d04a160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:58.482697 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6f778-5731-434d-a76c-b9700d04a160 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:58.482974 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6f778-5731-434d-a76c-b9700d04a160 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:58.483249 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6f778-5731-434d-a76c-b9700d04a160 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:58.483936 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6f778-5731-434d-a76c-b9700d04a160 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:58.484235 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6f778-5731-434d-a76c-b9700d04a160 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:58.485618 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6f778-5731-434d-a76c-b9700d04a160 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:58.495952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 244/480] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:54:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:58.865903 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331e0ba-c4ff-4167-875e-73ece00fb62d None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:58.866413 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331e0ba-c4ff-4167-875e-73ece00fb62d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:58.866413 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331e0ba-c4ff-4167-875e-73ece00fb62d None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:58.868045 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 237/481] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:54:58.873574 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae77122-4be5-4901-b204-e834e392ce86 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:58.874043 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae77122-4be5-4901-b204-e834e392ce86 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:58.874121 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae77122-4be5-4901-b204-e834e392ce86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:58.894885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-cae77122-4be5-4901-b204-e834e392ce86 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:54:58.896073 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cae77122-4be5-4901-b204-e834e392ce86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:54:58.986619 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 245/482] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:54:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:54:59.017169 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331e0ba-c4ff-4167-875e-73ece00fb62d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:59.146899 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331e0ba-c4ff-4167-875e-73ece00fb62d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:59.193860 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef165c87-f2a5-4072-b623-2c5690affa29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:59.194518 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef165c87-f2a5-4072-b623-2c5690affa29 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:59.194795 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef165c87-f2a5-4072-b623-2c5690affa29 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:59.195061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef165c87-f2a5-4072-b623-2c5690affa29 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:59.210210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef165c87-f2a5-4072-b623-2c5690affa29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:54:59.215624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-ef165c87-f2a5-4072-b623-2c5690affa29 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:54:59.215624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:59.215624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:59.215624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:59.215624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:59.215624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:59.215624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:59.215624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:59.215624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:59.215624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:59.215624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:59.215624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:59.215624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:54:59.215624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:54:59.215624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:54:59.215624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:54:59.216511 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:54:59.216511 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:54:59.216511 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:54:59.216511 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:54:59.216511 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:59.216511 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:59.216511 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:59.216511 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:59.216511 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:59.216511 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:59.216511 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:59.216511 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:59.216511 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:59.216511 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:59.216511 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:59.216511 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:59.217810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:59.217810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:59.217810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:59.217810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:59.217810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:54:59.217810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:54:59.217810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:54:59.217810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:54:59.217810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:54:59.217810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:54:59.217810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:54:59.217810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:59.217810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 238/483] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 10 16:54:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:59.224784 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae77122-4be5-4901-b204-e834e392ce86 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:59.322758 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:59.352255 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae77122-4be5-4901-b204-e834e392ce86 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:59.387759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e14900-4b94-4e92-90c1-a606eb68f467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:59.388364 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e14900-4b94-4e92-90c1-a606eb68f467 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:59.388642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e14900-4b94-4e92-90c1-a606eb68f467 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:59.388913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e14900-4b94-4e92-90c1-a606eb68f467 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:59.389577 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e14900-4b94-4e92-90c1-a606eb68f467 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:59.389876 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e14900-4b94-4e92-90c1-a606eb68f467 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:59.391393 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e14900-4b94-4e92-90c1-a606eb68f467 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:59.400121 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 246/484] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 10 16:54:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:59.411464 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef165c87-f2a5-4072-b623-2c5690affa29 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:59.458778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:59.554219 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef165c87-f2a5-4072-b623-2c5690affa29 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:59.607345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18debc1e-5d71-46bf-9268-12f84ea9786b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:59.607345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18debc1e-5d71-46bf-9268-12f84ea9786b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:59.607642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18debc1e-5d71-46bf-9268-12f84ea9786b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:59.607868 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18debc1e-5d71-46bf-9268-12f84ea9786b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:59.628536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18debc1e-5d71-46bf-9268-12f84ea9786b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:59.629986 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18debc1e-5d71-46bf-9268-12f84ea9786b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:59.655116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-18debc1e-5d71-46bf-9268-12f84ea9786b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:54:59.655116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:54:59.655116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:54:59.655116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:54:59.655116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:54:59.655116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:54:59.655116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:54:59.655116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:54:59.655116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:54:59.655116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:54:59.655116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:54:59.655116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:54:59.655116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:54:59.655116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:54:59.655116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:54:59.655116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:54:59.656039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:54:59.656039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:54:59.656039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:54:59.656039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:54:59.656039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:54:59.656039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:54:59.656039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:54:59.656039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:54:59.656039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:54:59.656039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:54:59.656039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:54:59.656039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:54:59.656039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:54:59.656039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:54:59.656039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:54:59.656039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:54:59.656039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:54:59.656876 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:54:59.656876 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:54:59.656876 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:54:59.656876 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 239/485] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:54:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:54:59.663103 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e14900-4b94-4e92-90c1-a606eb68f467 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:59.790167 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e14900-4b94-4e92-90c1-a606eb68f467 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:54:59.829401 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003835d-a4ef-43f1-8a77-e7d537f5f96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:54:59.830331 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003835d-a4ef-43f1-8a77-e7d537f5f96e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:54:59.830787 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003835d-a4ef-43f1-8a77-e7d537f5f96e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:54:59.831199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003835d-a4ef-43f1-8a77-e7d537f5f96e None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:54:59.831967 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3003835d-a4ef-43f1-8a77-e7d537f5f96e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:54:59.832409 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3003835d-a4ef-43f1-8a77-e7d537f5f96e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:54:59.834000 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3003835d-a4ef-43f1-8a77-e7d537f5f96e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:54:59.841618 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:59.850803 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 247/486] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:54:59.859447 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18debc1e-5d71-46bf-9268-12f84ea9786b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:54:59.978822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:54:59.995238 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18debc1e-5d71-46bf-9268-12f84ea9786b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:00.049763 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390706c-40bf-4166-ace2-29349abc71f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:00.050429 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390706c-40bf-4166-ace2-29349abc71f8 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:00.050772 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390706c-40bf-4166-ace2-29349abc71f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:00.051049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390706c-40bf-4166-ace2-29349abc71f8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:00.059342 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e390706c-40bf-4166-ace2-29349abc71f8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:00.060791 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e390706c-40bf-4166-ace2-29349abc71f8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:00.078095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-e390706c-40bf-4166-ace2-29349abc71f8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:55:00.078095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:00.078095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:00.078095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:00.078095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:00.078095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:00.078095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:00.078095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:00.078095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:00.078095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:00.078095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:00.078095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:00.078095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:00.078095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:00.078095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:00.078095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:00.079061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:00.079061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:00.079061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:00.079061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:00.079061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:00.079061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:00.079061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:00.079061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:00.079061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:00.079061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:00.079061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:00.079061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:00.079061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:00.079061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:00.079061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:00.079061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:00.079061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:00.079061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:00.079948 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:00.079948 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:00.079948 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:00.079948 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:00.079948 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:00.079948 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:00.079948 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:00.079948 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:00.079948 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:55:00.079948 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:00.079948 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 240/487] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 10 16:54:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:00.086821 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003835d-a4ef-43f1-8a77-e7d537f5f96e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:00.234727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003835d-a4ef-43f1-8a77-e7d537f5f96e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:00.281453 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04667aef-ba3d-4be1-bcf1-b78c27373031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:00.282063 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04667aef-ba3d-4be1-bcf1-b78c27373031 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:00.282339 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04667aef-ba3d-4be1-bcf1-b78c27373031 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:00.282631 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04667aef-ba3d-4be1-bcf1-b78c27373031 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:00.283362 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04667aef-ba3d-4be1-bcf1-b78c27373031 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:00.283670 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04667aef-ba3d-4be1-bcf1-b78c27373031 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:00.285118 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04667aef-ba3d-4be1-bcf1-b78c27373031 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:00.332118 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 248/488] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 10 16:55:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:00.339999 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390706c-40bf-4166-ace2-29349abc71f8 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:00.395866 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf03924-9d84-4e5a-977e-edc051da1dc3 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:00.396205 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf03924-9d84-4e5a-977e-edc051da1dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:00.396461 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf03924-9d84-4e5a-977e-edc051da1dc3 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:00.398055 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 249/489] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:00.407792 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c0552-85cc-4c29-84b3-faed066b1f20 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:00.408079 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c0552-85cc-4c29-84b3-faed066b1f20 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:00.408329 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c0552-85cc-4c29-84b3-faed066b1f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:00.429887 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bd8c0552-85cc-4c29-84b3-faed066b1f20 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:00.431458 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bd8c0552-85cc-4c29-84b3-faed066b1f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:00.502861 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:00.507608 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 250/490] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:55:00.518448 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390706c-40bf-4166-ace2-29349abc71f8 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:00.548366 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c0552-85cc-4c29-84b3-faed066b1f20 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:00.572748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0515b-dd4b-47bd-9c80-7dd26a2149d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:00.575865 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0515b-dd4b-47bd-9c80-7dd26a2149d1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:00.576350 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0515b-dd4b-47bd-9c80-7dd26a2149d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:00.576857 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0515b-dd4b-47bd-9c80-7dd26a2149d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:00.606777 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:00.661303 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0515b-dd4b-47bd-9c80-7dd26a2149d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:00.663120 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0515b-dd4b-47bd-9c80-7dd26a2149d1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:00.755749 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c0552-85cc-4c29-84b3-faed066b1f20 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:00.774236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 241/491] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:55:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:55:00.776100 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6abcc-1b54-4a56-b85a-9cd50f5277d1 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:00.776562 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6abcc-1b54-4a56-b85a-9cd50f5277d1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:00.777105 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6abcc-1b54-4a56-b85a-9cd50f5277d1 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:00.778811 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 242/492] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:00.779656 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6abcc-1b54-4a56-b85a-9cd50f5277d1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:00.794634 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11901f6-6aeb-4949-8cb3-5bf1f18325d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:00.795554 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11901f6-6aeb-4949-8cb3-5bf1f18325d6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:00.796027 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11901f6-6aeb-4949-8cb3-5bf1f18325d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:00.796436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11901f6-6aeb-4949-8cb3-5bf1f18325d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:00.807878 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11901f6-6aeb-4949-8cb3-5bf1f18325d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:00.813890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-b11901f6-6aeb-4949-8cb3-5bf1f18325d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:00.813890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:00.813890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:00.813890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:00.813890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:00.813890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:00.813890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:00.813890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:00.813890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:00.813890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:00.813890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:00.813890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:00.813890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:00.813890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:00.813890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:00.813890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:00.815089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:00.815089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:00.815089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:00.815089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:00.815089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:00.815089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:00.815089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:00.815089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:00.815089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:00.815089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:00.815089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:00.815089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:00.815089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:00.815089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:00.815089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:00.815089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:00.815966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:00.815966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:00.815966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:00.815966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:00.815966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:00.815966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:00.815966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:00.815966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:00.815966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:00.815966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:00.815966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:00.815966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:00.818176 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 251/493] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 10 16:55:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:00.821280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb54d28-740c-4a47-922a-98920c4cb08a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:00.821775 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb54d28-740c-4a47-922a-98920c4cb08a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:00.822182 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb54d28-740c-4a47-922a-98920c4cb08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:00.838431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:00.841527 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ecb54d28-740c-4a47-922a-98920c4cb08a None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:00.842937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ecb54d28-740c-4a47-922a-98920c4cb08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:00.891507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:00.897993 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6abcc-1b54-4a56-b85a-9cd50f5277d1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:00.920098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 252/494] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:55:00.921264 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb54d28-740c-4a47-922a-98920c4cb08a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:00.945937 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80198070-413c-4ec7-89c1-93a27f6d44c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:00.946595 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80198070-413c-4ec7-89c1-93a27f6d44c3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:00.946872 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80198070-413c-4ec7-89c1-93a27f6d44c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:00.947140 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80198070-413c-4ec7-89c1-93a27f6d44c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:00.958153 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80198070-413c-4ec7-89c1-93a27f6d44c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:00.959663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80198070-413c-4ec7-89c1-93a27f6d44c3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:00.975546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-80198070-413c-4ec7-89c1-93a27f6d44c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:00.975546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:00.975546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:00.975546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:00.975546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:00.975546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:00.975546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:00.975546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:00.975546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:00.975546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:00.975546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:00.975546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:00.975546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:00.975546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:00.975546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:00.975546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:00.976494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:00.976494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:00.976494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:00.976494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:00.976494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:00.976494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:00.976494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:00.976494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:00.976494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:00.976494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:00.976494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:00.976494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:00.976494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:00.976494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:00.976494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:00.976494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:00.976494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:00.977329 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:00.977329 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:00.977329 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:00.977329 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 243/495] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:00.978235 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645151ad-bf0b-4cee-988e-4a6bbafe5f5b None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:00.978548 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645151ad-bf0b-4cee-988e-4a6bbafe5f5b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:00.978819 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645151ad-bf0b-4cee-988e-4a6bbafe5f5b None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:00.980724 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 244/496] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:00.981627 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645151ad-bf0b-4cee-988e-4a6bbafe5f5b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:01.036180 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb54d28-740c-4a47-922a-98920c4cb08a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:01.071523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19599bbb-1813-4faa-88d4-ae42151d28c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:01.072169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19599bbb-1813-4faa-88d4-ae42151d28c8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:01.072554 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19599bbb-1813-4faa-88d4-ae42151d28c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:01.072723 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19599bbb-1813-4faa-88d4-ae42151d28c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:01.073381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19599bbb-1813-4faa-88d4-ae42151d28c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:01.074194 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19599bbb-1813-4faa-88d4-ae42151d28c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:01.075614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19599bbb-1813-4faa-88d4-ae42151d28c8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:01.077435 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:01.083887 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645151ad-bf0b-4cee-988e-4a6bbafe5f5b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:01.084324 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 253/497] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 10 16:55:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:01.085126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19599bbb-1813-4faa-88d4-ae42151d28c8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:01.118101 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733da44e-a229-47af-8def-c51ee8cc6536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:01.118729 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733da44e-a229-47af-8def-c51ee8cc6536 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:01.119001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733da44e-a229-47af-8def-c51ee8cc6536 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:01.119267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733da44e-a229-47af-8def-c51ee8cc6536 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:01.128342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:01.130673 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733da44e-a229-47af-8def-c51ee8cc6536 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:01.135273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-733da44e-a229-47af-8def-c51ee8cc6536 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:01.135273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:01.135273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:01.135273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:01.135273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:01.135273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:01.135273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:01.135273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:01.135273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:01.135273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:01.135273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:01.135273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:01.135273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:01.135273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:01.135273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:01.135273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:01.140998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:01.140998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:01.140998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:01.140998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:01.140998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:01.140998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:01.140998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:01.140998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:01.140998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:01.140998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:01.140998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:01.140998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:01.140998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:01.140998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:01.140998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:01.140998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:01.141783 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:01.141783 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:01.141783 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:01.141783 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:01.141783 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:01.141783 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:01.141783 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:01.141783 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:01.141783 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:01.141783 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:01.141783 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:01.141783 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:01.141783 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 245/498] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 10 16:55:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:01.141783 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e36a05-8a00-4294-bf0b-fbe0a63e1369 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:01.142567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e36a05-8a00-4294-bf0b-fbe0a63e1369 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:01.142567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e36a05-8a00-4294-bf0b-fbe0a63e1369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:01.156967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-42e36a05-8a00-4294-bf0b-fbe0a63e1369 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:01.158187 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-42e36a05-8a00-4294-bf0b-fbe0a63e1369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:01.190906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19599bbb-1813-4faa-88d4-ae42151d28c8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:01.231195 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f29000-e3bc-45cb-9520-e03e9abb7ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:01.231450 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 246/499] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:55:01.231531 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f29000-e3bc-45cb-9520-e03e9abb7ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:01.231531 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f29000-e3bc-45cb-9520-e03e9abb7ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:01.231531 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f29000-e3bc-45cb-9520-e03e9abb7ff6 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:01.231735 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf6bb1-9dae-4b39-81c1-4bdde3db0a9e None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:01.231735 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf6bb1-9dae-4b39-81c1-4bdde3db0a9e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:01.231853 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f29000-e3bc-45cb-9520-e03e9abb7ff6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:01.231928 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf6bb1-9dae-4b39-81c1-4bdde3db0a9e None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:01.231996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f29000-e3bc-45cb-9520-e03e9abb7ff6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:01.232070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 247/500] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:01.232138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f29000-e3bc-45cb-9520-e03e9abb7ff6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:01.232212 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf6bb1-9dae-4b39-81c1-4bdde3db0a9e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:01.243088 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 254/501] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:01.244157 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f29000-e3bc-45cb-9520-e03e9abb7ff6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:01.331038 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:01.342180 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:01.346233 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f29000-e3bc-45cb-9520-e03e9abb7ff6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:01.347883 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf6bb1-9dae-4b39-81c1-4bdde3db0a9e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:01.380021 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b64c4-2aca-4b6f-a7ba-37f81e59ab12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:01.380708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b64c4-2aca-4b6f-a7ba-37f81e59ab12 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:01.380994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b64c4-2aca-4b6f-a7ba-37f81e59ab12 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:01.381266 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b64c4-2aca-4b6f-a7ba-37f81e59ab12 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:01.381927 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b64c4-2aca-4b6f-a7ba-37f81e59ab12 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:01.382228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b64c4-2aca-4b6f-a7ba-37f81e59ab12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:01.383644 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b64c4-2aca-4b6f-a7ba-37f81e59ab12 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:01.384821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9dd49f-248b-47fa-9843-f89513272780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:01.385342 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9dd49f-248b-47fa-9843-f89513272780 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:01.385697 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9dd49f-248b-47fa-9843-f89513272780 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:01.385954 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9dd49f-248b-47fa-9843-f89513272780 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:01.392751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 255/502] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 10 16:55:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:01.393943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdaaa3e-bebb-4dc5-ab19-7b41d701e4a2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:01.394743 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdaaa3e-bebb-4dc5-ab19-7b41d701e4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:01.395018 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdaaa3e-bebb-4dc5-ab19-7b41d701e4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:01.401306 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9dd49f-248b-47fa-9843-f89513272780 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:01.402923 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9dd49f-248b-47fa-9843-f89513272780 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:01.412457 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3bdaaa3e-bebb-4dc5-ab19-7b41d701e4a2 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:01.413641 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3bdaaa3e-bebb-4dc5-ab19-7b41d701e4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:01.417958 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-bd9dd49f-248b-47fa-9843-f89513272780 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:01.417958 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:01.417958 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:01.417958 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:01.417958 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:01.417958 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:01.417958 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:01.417958 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:01.417958 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:01.417958 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:01.417958 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:01.417958 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:01.417958 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:01.417958 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:01.417958 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:01.417958 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:01.418936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:01.418936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:01.418936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:01.418936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:01.418936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:01.418936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:01.418936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:01.418936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:01.418936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:01.418936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:01.418936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:01.418936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:01.418936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:01.418936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:01.418936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:01.418936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:01.418936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:01.419822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:01.419822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:01.419822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:01.419822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 248/503] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:01.420036 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9dd49f-248b-47fa-9843-f89513272780 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:01.479156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 256/504] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:55:01.480544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdaaa3e-bebb-4dc5-ab19-7b41d701e4a2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:01.535846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9dd49f-248b-47fa-9843-f89513272780 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:01.569401 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b597a0e-e93b-45e0-99ce-8c4f8ac9979b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:01.570009 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b597a0e-e93b-45e0-99ce-8c4f8ac9979b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:01.570281 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b597a0e-e93b-45e0-99ce-8c4f8ac9979b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:01.570572 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b597a0e-e93b-45e0-99ce-8c4f8ac9979b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:01.577157 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b597a0e-e93b-45e0-99ce-8c4f8ac9979b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:01.577614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:01.578627 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b597a0e-e93b-45e0-99ce-8c4f8ac9979b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:01.580304 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:01.594483 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdaaa3e-bebb-4dc5-ab19-7b41d701e4a2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:01.594663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-4b597a0e-e93b-45e0-99ce-8c4f8ac9979b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:01.594663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:01.594663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:01.594663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:01.594663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:01.594663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:01.594663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:01.594663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:01.594663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:01.594663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:01.594663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:01.594663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:01.594663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:01.594663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:01.594663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:01.595474 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:01.595474 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:01.595474 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:01.595474 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:01.595474 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:01.595474 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:01.595474 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:01.595474 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:01.595474 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:01.595474 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:01.595474 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:01.595474 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:01.595474 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:01.595474 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:01.595474 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:01.595474 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:01.595474 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:01.596322 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:01.596322 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:01.596322 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:01.596322 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:01.596322 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:01.596322 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:01.596322 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:01.596322 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:01.596322 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 249/505] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:01.596322 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b597a0e-e93b-45e0-99ce-8c4f8ac9979b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:01.625398 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090ee97-2d52-4bf9-9e29-a9632a7f3a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:01.626031 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090ee97-2d52-4bf9-9e29-a9632a7f3a51 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:01.626295 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090ee97-2d52-4bf9-9e29-a9632a7f3a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:01.626601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090ee97-2d52-4bf9-9e29-a9632a7f3a51 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:01.636714 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090ee97-2d52-4bf9-9e29-a9632a7f3a51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:01.641421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2090ee97-2d52-4bf9-9e29-a9632a7f3a51 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:01.641421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:01.641421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:01.641421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:01.641421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:01.641421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:01.641421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:01.641421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:01.641421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:01.641421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:01.641421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:01.641421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:01.641421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:01.641421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:01.641421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:01.641421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:01.642280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:01.642280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:01.642280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:01.642280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:01.642280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:01.642280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:01.642280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:01.642280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:01.642280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:01.642280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:01.642280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:01.642280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:01.642280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:01.642280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:01.642280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:01.642280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:01.643135 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:01.643135 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:01.643135 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:01.643135 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:01.643135 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:01.643135 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:01.643135 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:01.643135 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:01.643135 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:01.643135 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:01.643135 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:01.643135 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:01.643135 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 257/506] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 10 16:55:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:01.644094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090ee97-2d52-4bf9-9e29-a9632a7f3a51 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:01.697480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b597a0e-e93b-45e0-99ce-8c4f8ac9979b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:01.730988 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45e901-facb-4a7a-80fe-10221bffec9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:01.731594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45e901-facb-4a7a-80fe-10221bffec9a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:01.731871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45e901-facb-4a7a-80fe-10221bffec9a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:01.732136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45e901-facb-4a7a-80fe-10221bffec9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:01.742401 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de45e901-facb-4a7a-80fe-10221bffec9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:01.743854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de45e901-facb-4a7a-80fe-10221bffec9a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:01.744535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090ee97-2d52-4bf9-9e29-a9632a7f3a51 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:01.758331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-de45e901-facb-4a7a-80fe-10221bffec9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:01.758331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:01.758331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:01.758331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:01.758331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:01.758331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:01.758331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:01.758331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:01.758331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:01.758331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:01.758331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:01.758331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:01.758331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:01.758331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:01.758331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:01.758331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:01.759336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:01.759336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:01.759336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:01.759336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:01.759336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:01.759336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:01.759336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:01.759336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:01.759336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:01.759336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:01.759336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:01.759336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:01.759336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:01.759336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:01.759336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:01.759336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:01.759336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:01.760168 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:01.760168 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:01.760168 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:01.760168 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 250/507] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:01.760391 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45e901-facb-4a7a-80fe-10221bffec9a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:01.781541 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be96606-ca00-4c43-a8d6-721f8a80f589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:01.782156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be96606-ca00-4c43-a8d6-721f8a80f589 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:01.782463 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be96606-ca00-4c43-a8d6-721f8a80f589 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:01.782733 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be96606-ca00-4c43-a8d6-721f8a80f589 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:01.783391 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be96606-ca00-4c43-a8d6-721f8a80f589 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:01.783686 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be96606-ca00-4c43-a8d6-721f8a80f589 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:01.785069 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be96606-ca00-4c43-a8d6-721f8a80f589 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:01.802636 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 258/508] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:01.802636 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be96606-ca00-4c43-a8d6-721f8a80f589 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:01.812614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:01.818917 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:01.869058 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45e901-facb-4a7a-80fe-10221bffec9a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:01.901444 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be96606-ca00-4c43-a8d6-721f8a80f589 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:01.902930 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037def4b-ec5c-4538-a4b2-3d51a9c50978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:01.903605 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037def4b-ec5c-4538-a4b2-3d51a9c50978 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:01.903817 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037def4b-ec5c-4538-a4b2-3d51a9c50978 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:01.904079 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037def4b-ec5c-4538-a4b2-3d51a9c50978 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:01.914246 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037def4b-ec5c-4538-a4b2-3d51a9c50978 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:01.915790 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037def4b-ec5c-4538-a4b2-3d51a9c50978 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:01.930049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-037def4b-ec5c-4538-a4b2-3d51a9c50978 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:01.930049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:01.930049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:01.930049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:01.930049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:01.930049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:01.930049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:01.930049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:01.930049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:01.930049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:01.930049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:01.930049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:01.930049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:01.930049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:01.930049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:01.930049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:01.930996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:01.930996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:01.930996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:01.930996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:01.930996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:01.930996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:01.930996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:01.930996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:01.930996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:01.930996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:01.930996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:01.930996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:01.930996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:01.930996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:01.930996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:01.930996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:01.930996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:01.931888 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:01.931888 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:01.931888 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:01.931888 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 251/509] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:01.932101 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037def4b-ec5c-4538-a4b2-3d51a9c50978 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:01.939233 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc609112-6cb8-48d6-89f8-0895fd4ce6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:01.939837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc609112-6cb8-48d6-89f8-0895fd4ce6de None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:01.940112 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc609112-6cb8-48d6-89f8-0895fd4ce6de None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:01.940415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc609112-6cb8-48d6-89f8-0895fd4ce6de None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:01.941101 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc609112-6cb8-48d6-89f8-0895fd4ce6de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:01.941407 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc609112-6cb8-48d6-89f8-0895fd4ce6de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:01.942875 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc609112-6cb8-48d6-89f8-0895fd4ce6de None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:01.960169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 259/510] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:01.960974 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc609112-6cb8-48d6-89f8-0895fd4ce6de None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:02.028485 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:02.033419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037def4b-ec5c-4538-a4b2-3d51a9c50978 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:02.034954 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:02.057583 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc609112-6cb8-48d6-89f8-0895fd4ce6de None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:02.067240 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb89c1f-0d7c-4083-b71e-525040fe1067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:02.067832 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb89c1f-0d7c-4083-b71e-525040fe1067 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:02.068109 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb89c1f-0d7c-4083-b71e-525040fe1067 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:02.068372 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb89c1f-0d7c-4083-b71e-525040fe1067 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:02.069025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb89c1f-0d7c-4083-b71e-525040fe1067 None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:02.069431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb89c1f-0d7c-4083-b71e-525040fe1067 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:02.070843 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb89c1f-0d7c-4083-b71e-525040fe1067 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:02.079419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 252/511] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 10 16:55:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:02.080236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb89c1f-0d7c-4083-b71e-525040fe1067 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:02.096337 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b85f1-8ab8-426b-977f-5ee161fc0902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:02.096944 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b85f1-8ab8-426b-977f-5ee161fc0902 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:02.097221 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b85f1-8ab8-426b-977f-5ee161fc0902 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:02.097490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b85f1-8ab8-426b-977f-5ee161fc0902 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:02.098128 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b85f1-8ab8-426b-977f-5ee161fc0902 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:02.098454 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b85f1-8ab8-426b-977f-5ee161fc0902 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:02.099835 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b85f1-8ab8-426b-977f-5ee161fc0902 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:02.111666 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 260/512] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:02.112462 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b85f1-8ab8-426b-977f-5ee161fc0902 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:02.180748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb89c1f-0d7c-4083-b71e-525040fe1067 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:02.214110 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b85f1-8ab8-426b-977f-5ee161fc0902 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:02.215210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b18647-5d90-4a44-a412-e7496f2e9f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:02.215563 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b18647-5d90-4a44-a412-e7496f2e9f2a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:02.215843 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b18647-5d90-4a44-a412-e7496f2e9f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:02.216114 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b18647-5d90-4a44-a412-e7496f2e9f2a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:02.223325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b18647-5d90-4a44-a412-e7496f2e9f2a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:02.224746 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b18647-5d90-4a44-a412-e7496f2e9f2a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:02.235591 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-f7b18647-5d90-4a44-a412-e7496f2e9f2a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 16:55:02.235591 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:02.235591 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:02.235591 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:02.235591 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:02.235591 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:02.235591 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:02.235591 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:02.235591 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:02.235591 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:02.235591 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:02.235591 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:02.235591 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:02.235591 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:02.235591 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:02.235591 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:02.236674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:02.236674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:02.236674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:02.236674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:02.236674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:02.236674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:02.236674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:02.236674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:02.236674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:02.236674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:02.236674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:02.236674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:02.236674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:02.236674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:02.236674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:02.236674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:02.236674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:02.236674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:02.237805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:02.237805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:02.237805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:02.237805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:02.237805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:02.237805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:02.237805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:02.237805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:02.237805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 16:55:02.237805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:02.237805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 253/513] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 10 16:55:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:02.237805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b18647-5d90-4a44-a412-e7496f2e9f2a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:02.244966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:02.251180 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c56182-4e58-47fd-82e5-05e08faf065c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:02.255558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c56182-4e58-47fd-82e5-05e08faf065c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:02.255558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c56182-4e58-47fd-82e5-05e08faf065c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:02.255558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c56182-4e58-47fd-82e5-05e08faf065c None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:02.255558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c56182-4e58-47fd-82e5-05e08faf065c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:02.255558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c56182-4e58-47fd-82e5-05e08faf065c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:02.255558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c56182-4e58-47fd-82e5-05e08faf065c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:02.263882 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:02.266947 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 261/514] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:02.267878 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c56182-4e58-47fd-82e5-05e08faf065c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:02.342911 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b18647-5d90-4a44-a412-e7496f2e9f2a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:02.368001 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c56182-4e58-47fd-82e5-05e08faf065c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:02.378239 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e85cd8-d2b8-4ab3-b7a1-8b2582b7fa2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:02.379159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e85cd8-d2b8-4ab3-b7a1-8b2582b7fa2f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:02.379434 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e85cd8-d2b8-4ab3-b7a1-8b2582b7fa2f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:02.379703 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e85cd8-d2b8-4ab3-b7a1-8b2582b7fa2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:02.380417 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e85cd8-d2b8-4ab3-b7a1-8b2582b7fa2f None admin] RBAC: Adding query filter params (role.id=303e0fba379a41019ecc2e119d87fb1b, scope.project.id=d11f33af53374dffa61132642a7770f4, user.id=615cc36e21cc493b823e6cee695ac308) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:02.380716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e85cd8-d2b8-4ab3-b7a1-8b2582b7fa2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:02.382153 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e85cd8-d2b8-4ab3-b7a1-8b2582b7fa2f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:02.387875 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 254/515] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:55:02 2022] GET /identity/v3/role_assignments?user.id=615cc36e21cc493b823e6cee695ac308&scope.project.id=d11f33af53374dffa61132642a7770f4&role.id=303e0fba379a41019ecc2e119d87fb1b => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:02.388652 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e85cd8-d2b8-4ab3-b7a1-8b2582b7fa2f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:02.406904 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c0416-4c90-49f3-a5b7-dc6cbb76970c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:02.407849 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c0416-4c90-49f3-a5b7-dc6cbb76970c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:02.408282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c0416-4c90-49f3-a5b7-dc6cbb76970c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:02.408552 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c0416-4c90-49f3-a5b7-dc6cbb76970c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:02.420364 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c0416-4c90-49f3-a5b7-dc6cbb76970c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:02.421875 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c0416-4c90-49f3-a5b7-dc6cbb76970c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:02.438601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-e69c0416-4c90-49f3-a5b7-dc6cbb76970c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:02.438601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:02.438601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:02.438601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:02.438601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:02.438601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:02.438601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:02.438601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:02.438601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:02.438601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:02.438601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:02.438601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:02.438601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:02.438601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:02.438601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:02.438601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:02.439534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:02.439534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:02.439534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:02.439534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:02.439534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:02.439534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:02.439534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:02.439534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:02.439534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:02.439534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:02.439534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:02.439534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:02.439534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:02.439534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:02.439534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:02.439534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:02.439534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:02.440363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:02.440363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:02.440363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:02.440363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 262/516] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:02.441009 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c0416-4c90-49f3-a5b7-dc6cbb76970c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:02.493223 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:02.494087 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:02.506705 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e85cd8-d2b8-4ab3-b7a1-8b2582b7fa2f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:02.545498 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f332d35-3306-491b-b4bb-3d7634507d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:02.546962 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f332d35-3306-491b-b4bb-3d7634507d6a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:02.548191 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f332d35-3306-491b-b4bb-3d7634507d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:02.548545 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f332d35-3306-491b-b4bb-3d7634507d6a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:02.556020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c0416-4c90-49f3-a5b7-dc6cbb76970c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:02.557200 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f332d35-3306-491b-b4bb-3d7634507d6a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:02.564607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f332d35-3306-491b-b4bb-3d7634507d6a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:02.570737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-9f332d35-3306-491b-b4bb-3d7634507d6a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 16:55:02.570737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:02.570737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:02.570737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:02.570737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:02.570737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:02.570737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:02.570737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:02.570737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:02.570737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:02.570737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:02.570737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:02.570737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:02.570737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:02.570737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:02.570737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:02.571645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:02.571645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:02.571645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:02.571645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:02.571645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:02.571645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:02.571645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:02.571645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:02.571645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:02.571645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:02.571645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:02.571645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:02.571645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:02.571645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:02.571645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:02.571645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:02.571645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:02.571645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:02.572536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:02.572536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:02.572536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:02.572536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:02.572536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:02.572536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:02.572536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:02.572536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:02.572536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 16:55:02.572536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:02.572536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 255/517] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:55:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:02.573090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f332d35-3306-491b-b4bb-3d7634507d6a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:02.599000 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9bf80b-27a3-4d7d-a5fa-76c1f7883c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:02.599604 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bf80b-27a3-4d7d-a5fa-76c1f7883c1c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:02.599893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bf80b-27a3-4d7d-a5fa-76c1f7883c1c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:02.600152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bf80b-27a3-4d7d-a5fa-76c1f7883c1c None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:02.600878 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9bf80b-27a3-4d7d-a5fa-76c1f7883c1c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:02.601182 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9bf80b-27a3-4d7d-a5fa-76c1f7883c1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:02.602648 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9bf80b-27a3-4d7d-a5fa-76c1f7883c1c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:02.625496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 263/518] 10.222.0.45 () {56 vars in 1139 bytes} [Wed Aug 10 16:55:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:02.626340 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9bf80b-27a3-4d7d-a5fa-76c1f7883c1c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:02.687596 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f332d35-3306-491b-b4bb-3d7634507d6a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:02.724992 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536db88-c3d9-44f9-92dc-756304803863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:02.725508 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536db88-c3d9-44f9-92dc-756304803863 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:02.725785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536db88-c3d9-44f9-92dc-756304803863 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:02.726059 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536db88-c3d9-44f9-92dc-756304803863 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:02.726723 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8536db88-c3d9-44f9-92dc-756304803863 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:02.728149 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8536db88-c3d9-44f9-92dc-756304803863 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:02.731873 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:02.734968 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9bf80b-27a3-4d7d-a5fa-76c1f7883c1c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:02.769545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e31bdd-c5f9-4053-b012-599388107a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:02.770738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e31bdd-c5f9-4053-b012-599388107a80 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:02.770738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e31bdd-c5f9-4053-b012-599388107a80 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:02.770738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e31bdd-c5f9-4053-b012-599388107a80 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:02.771359 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e31bdd-c5f9-4053-b012-599388107a80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:02.771653 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e31bdd-c5f9-4053-b012-599388107a80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:02.773005 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e31bdd-c5f9-4053-b012-599388107a80 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:02.783876 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 264/519] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:02.784645 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e31bdd-c5f9-4053-b012-599388107a80 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:02.886384 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e31bdd-c5f9-4053-b012-599388107a80 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:02.887115 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 256/520] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 10 16:55:02 2022] POST /identity/v3/users => generated 247 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:02.888804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536db88-c3d9-44f9-92dc-756304803863 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:02.903066 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:02.930539 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b259b9-af2e-4793-9437-e88389afb1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:02.931429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b259b9-af2e-4793-9437-e88389afb1e2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:02.931877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b259b9-af2e-4793-9437-e88389afb1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:02.932295 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b259b9-af2e-4793-9437-e88389afb1e2 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:02.933712 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b259b9-af2e-4793-9437-e88389afb1e2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:02.934211 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b259b9-af2e-4793-9437-e88389afb1e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:02.935834 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b259b9-af2e-4793-9437-e88389afb1e2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:02.958164 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 265/521] 10.222.0.45 () {56 vars in 1137 bytes} [Wed Aug 10 16:55:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:02.959393 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b259b9-af2e-4793-9437-e88389afb1e2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:02.982300 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:03.051340 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536db88-c3d9-44f9-92dc-756304803863 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:03.070851 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b259b9-af2e-4793-9437-e88389afb1e2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:03.094146 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed71cb-51e0-4083-b1c5-345a85e84b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:03.094784 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed71cb-51e0-4083-b1c5-345a85e84b74 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:03.095054 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed71cb-51e0-4083-b1c5-345a85e84b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:03.095321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed71cb-51e0-4083-b1c5-345a85e84b74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:03.116150 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7f996-4f2f-4958-b056-aae98537618a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:03.117191 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7f996-4f2f-4958-b056-aae98537618a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:03.117191 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7f996-4f2f-4958-b056-aae98537618a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:03.117408 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7f996-4f2f-4958-b056-aae98537618a None admin] PATH_INFO: `/v3/users/glance` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:03.127763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e7f996-4f2f-4958-b056-aae98537618a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:03.129220 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e7f996-4f2f-4958-b056-aae98537618a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:03.143276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-52e7f996-4f2f-4958-b056-aae98537618a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 16:55:03.143276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:03.143276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:03.143276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:03.143276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:03.143276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:03.143276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:03.143276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:03.143276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:03.143276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:03.143276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:03.143276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:03.143276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:03.143276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:03.143276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:03.143276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:03.144369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:03.144369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:03.144369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:03.144369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:03.144369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:03.144369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:03.144369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:03.144369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:03.144369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:03.144369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:03.144369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:03.144369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:03.144369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:03.144369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:03.144369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:03.144369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:03.144369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:03.144369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:03.145269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:03.145269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:03.145269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:03.145269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:03.145269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:03.145269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:03.145269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:03.145269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:03.145269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 16:55:03.145269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:03.145269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 266/522] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:55:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:03.145269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7f996-4f2f-4958-b056-aae98537618a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:03.180361 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efed71cb-51e0-4083-b1c5-345a85e84b74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:03.182497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efed71cb-51e0-4083-b1c5-345a85e84b74 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:03.187218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:03.257445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:03.281675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7f996-4f2f-4958-b056-aae98537618a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:03.304381 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 257/523] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:55:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:55:03.304381 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed71cb-51e0-4083-b1c5-345a85e84b74 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:03.322679 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ecd70-5a11-4627-8075-225721b52c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:03.323502 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ecd70-5a11-4627-8075-225721b52c2e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:03.324202 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ecd70-5a11-4627-8075-225721b52c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:03.324812 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ecd70-5a11-4627-8075-225721b52c2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:03.397516 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ecd70-5a11-4627-8075-225721b52c2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:03.399614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ecd70-5a11-4627-8075-225721b52c2e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:03.404983 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed71cb-51e0-4083-b1c5-345a85e84b74 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:03.407497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:03.441103 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfdf07-5639-4b19-ab61-75526f192701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:03.441895 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfdf07-5639-4b19-ab61-75526f192701 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:03.442268 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfdf07-5639-4b19-ab61-75526f192701 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:03.442669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfdf07-5639-4b19-ab61-75526f192701 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:03.443510 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfdf07-5639-4b19-ab61-75526f192701 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:03.443903 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfdf07-5639-4b19-ab61-75526f192701 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:03.445532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfdf07-5639-4b19-ab61-75526f192701 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:03.485441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 258/524] 10.222.0.45 () {56 vars in 1137 bytes} [Wed Aug 10 16:55:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:03.490401 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfdf07-5639-4b19-ab61-75526f192701 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:03.497319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:03.521547 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 267/525] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:55:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:55:03.522462 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ecd70-5a11-4627-8075-225721b52c2e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:03.594867 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfdf07-5639-4b19-ab61-75526f192701 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:03.631019 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad68fea-175e-4936-a3eb-5fdfad41a242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:03.631779 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ecd70-5a11-4627-8075-225721b52c2e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:03.631902 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad68fea-175e-4936-a3eb-5fdfad41a242 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:03.632220 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad68fea-175e-4936-a3eb-5fdfad41a242 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:03.632582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad68fea-175e-4936-a3eb-5fdfad41a242 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:03.643842 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad68fea-175e-4936-a3eb-5fdfad41a242 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:03.645429 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad68fea-175e-4936-a3eb-5fdfad41a242 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:03.661232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-cad68fea-175e-4936-a3eb-5fdfad41a242 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:03.661232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:03.661232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:03.661232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:03.661232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:03.661232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:03.661232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:03.661232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:03.661232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:03.661232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:03.661232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:03.661232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:03.661232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:03.661232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:03.661232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:03.661232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:03.662166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:03.662166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:03.662166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:03.662166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:03.662166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:03.662166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:03.662166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:03.662166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:03.662166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:03.662166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:03.662166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:03.662166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:03.662166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:03.662166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:03.662166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:03.662166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:03.662166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:03.663042 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:03.663042 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:03.663042 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:03.663042 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 259/526] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:03.663833 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad68fea-175e-4936-a3eb-5fdfad41a242 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:03.675121 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752fa48-6015-4c2c-bad9-a81f1e6b516d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:03.675784 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752fa48-6015-4c2c-bad9-a81f1e6b516d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:03.676062 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752fa48-6015-4c2c-bad9-a81f1e6b516d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:03.676331 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752fa48-6015-4c2c-bad9-a81f1e6b516d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:03.681305 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:03.726877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:03.750319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752fa48-6015-4c2c-bad9-a81f1e6b516d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:03.752149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752fa48-6015-4c2c-bad9-a81f1e6b516d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:03.768118 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad68fea-175e-4936-a3eb-5fdfad41a242 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:03.803857 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee4210-0d42-45bb-a629-6c9b8f0285e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:03.804424 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee4210-0d42-45bb-a629-6c9b8f0285e9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:03.804696 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee4210-0d42-45bb-a629-6c9b8f0285e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:03.804963 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee4210-0d42-45bb-a629-6c9b8f0285e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:03.815466 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee4210-0d42-45bb-a629-6c9b8f0285e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:03.816954 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee4210-0d42-45bb-a629-6c9b8f0285e9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:03.831880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-44ee4210-0d42-45bb-a629-6c9b8f0285e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:03.831880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:03.831880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:03.831880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:03.831880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:03.831880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:03.831880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:03.831880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:03.831880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:03.831880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:03.831880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:03.831880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:03.831880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:03.831880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:03.831880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:03.831880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:03.833039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:03.833039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:03.833039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:03.833039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:03.833039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:03.833039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:03.833039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:03.833039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:03.833039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:03.833039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:03.833039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:03.833039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:03.833039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:03.833039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:03.833039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:03.833039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:03.833039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:03.834026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:03.834026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:03.834026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:03.834026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 260/527] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:03.834026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee4210-0d42-45bb-a629-6c9b8f0285e9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:03.864396 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 268/528] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:55:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:55:03.865518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752fa48-6015-4c2c-bad9-a81f1e6b516d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:03.901254 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:03.936488 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee4210-0d42-45bb-a629-6c9b8f0285e9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:03.950339 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:03.972408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b4335-6e7f-486f-ae47-b348aeb71717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:03.973004 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b4335-6e7f-486f-ae47-b348aeb71717 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:03.973278 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b4335-6e7f-486f-ae47-b348aeb71717 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:03.973540 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b4335-6e7f-486f-ae47-b348aeb71717 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:03.974179 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b4335-6e7f-486f-ae47-b348aeb71717 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:03.974522 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b4335-6e7f-486f-ae47-b348aeb71717 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:03.975638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752fa48-6015-4c2c-bad9-a81f1e6b516d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:03.975886 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b4335-6e7f-486f-ae47-b348aeb71717 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:03.992566 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 261/529] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:03.992566 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b4335-6e7f-486f-ae47-b348aeb71717 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:04.021575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbfb8d1-0150-42be-b7dc-216a1858c59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:04.022528 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbfb8d1-0150-42be-b7dc-216a1858c59b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:04.023106 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbfb8d1-0150-42be-b7dc-216a1858c59b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:04.023694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbfb8d1-0150-42be-b7dc-216a1858c59b None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:04.024786 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbfb8d1-0150-42be-b7dc-216a1858c59b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:04.025263 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbfb8d1-0150-42be-b7dc-216a1858c59b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:04.027878 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbfb8d1-0150-42be-b7dc-216a1858c59b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:04.040841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 269/530] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:04.042160 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbfb8d1-0150-42be-b7dc-216a1858c59b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:04.116177 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b4335-6e7f-486f-ae47-b348aeb71717 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:04.146061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:04.157238 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71bfaaa-ec4e-4f93-a375-5f9f1d6654ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:04.158086 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bfaaa-ec4e-4f93-a375-5f9f1d6654ec None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:04.158536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bfaaa-ec4e-4f93-a375-5f9f1d6654ec None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:04.158994 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bfaaa-ec4e-4f93-a375-5f9f1d6654ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:04.162889 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbfb8d1-0150-42be-b7dc-216a1858c59b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:04.171190 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71bfaaa-ec4e-4f93-a375-5f9f1d6654ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:04.174301 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71bfaaa-ec4e-4f93-a375-5f9f1d6654ec None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:04.191338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-e71bfaaa-ec4e-4f93-a375-5f9f1d6654ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:04.191338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:04.191338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:04.191338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:04.191338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:04.191338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:04.191338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:04.191338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:04.191338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:04.191338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:04.191338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:04.191338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:04.191338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:04.191338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:04.191338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:04.191338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:04.192526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:04.192526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:04.192526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:04.192526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:04.192526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:04.192526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:04.192526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:04.192526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:04.192526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:04.192526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:04.192526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:04.192526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:04.192526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:04.192526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:04.192526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:04.192526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:04.192526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:04.193499 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:04.193499 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:04.193499 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:04.195075 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 262/531] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:04.196057 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71bfaaa-ec4e-4f93-a375-5f9f1d6654ec None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:04.204164 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead9ccd-465d-403b-909b-4e1088d46157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:04.204801 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead9ccd-465d-403b-909b-4e1088d46157 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:04.205089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead9ccd-465d-403b-909b-4e1088d46157 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:04.205354 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead9ccd-465d-403b-909b-4e1088d46157 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:04.214675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aead9ccd-465d-403b-909b-4e1088d46157 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:04.216722 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aead9ccd-465d-403b-909b-4e1088d46157 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:04.219064 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:04.234933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-aead9ccd-465d-403b-909b-4e1088d46157 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:04.234933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:04.234933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:04.234933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:04.234933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:04.234933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:04.234933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:04.234933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:04.234933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:04.234933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:04.234933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:04.234933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:04.234933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:04.234933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:04.234933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:04.242078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:04.242078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:04.242078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:04.242078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:04.242078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:04.242078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:04.242078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:04.242078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:04.242078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:04.242078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:04.242078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:04.242078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:04.242078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:04.242078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:04.242078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:04.242078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:04.242078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:04.243184 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:04.243184 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:04.243184 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:04.243184 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:04.243184 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:04.243184 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:04.243184 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:04.243184 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:04.243184 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 270/532] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:04.243184 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead9ccd-465d-403b-909b-4e1088d46157 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:04.330498 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71bfaaa-ec4e-4f93-a375-5f9f1d6654ec None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:04.371628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead9ccd-465d-403b-909b-4e1088d46157 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:04.380047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872baa5-f40e-42a3-b13a-4f155ac18dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:04.380792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872baa5-f40e-42a3-b13a-4f155ac18dee None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:04.381593 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872baa5-f40e-42a3-b13a-4f155ac18dee None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:04.381971 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872baa5-f40e-42a3-b13a-4f155ac18dee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:04.392358 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7872baa5-f40e-42a3-b13a-4f155ac18dee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:04.394505 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7872baa5-f40e-42a3-b13a-4f155ac18dee None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:04.409081 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd05700-80ae-4afc-83ea-aa7123cd2fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:04.409753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd05700-80ae-4afc-83ea-aa7123cd2fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:04.410037 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd05700-80ae-4afc-83ea-aa7123cd2fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:04.410306 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd05700-80ae-4afc-83ea-aa7123cd2fb6 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:04.410496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-7872baa5-f40e-42a3-b13a-4f155ac18dee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:04.410496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:04.410496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:04.410496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:04.410496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:04.410496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:04.410496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:04.410496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:04.410496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:04.410496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:04.410496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:04.410496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:04.410496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:04.410496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:04.410496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:04.411477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:04.411477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:04.411477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:04.411477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:04.411477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:04.411477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:04.411477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:04.411477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:04.411477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:04.411477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:04.411477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:04.411477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:04.411477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:04.411477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:04.411477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:04.411477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:04.411477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:04.412554 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:04.412554 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:04.412554 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:04.412554 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:04.412554 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:04.412554 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:04.412554 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:04.412554 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:04.413072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd05700-80ae-4afc-83ea-aa7123cd2fb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:04.413072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd05700-80ae-4afc-83ea-aa7123cd2fb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:04.413072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd05700-80ae-4afc-83ea-aa7123cd2fb6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:04.414632 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 263/533] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:04.415498 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872baa5-f40e-42a3-b13a-4f155ac18dee None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:04.430517 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 271/534] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:04.431704 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd05700-80ae-4afc-83ea-aa7123cd2fb6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:04.466399 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:04.499645 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:04.531012 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872baa5-f40e-42a3-b13a-4f155ac18dee None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:04.541117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd05700-80ae-4afc-83ea-aa7123cd2fb6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:04.568027 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fedb53a-cafd-4e36-ba8d-a7754c96dc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:04.568626 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fedb53a-cafd-4e36-ba8d-a7754c96dc78 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:04.569061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fedb53a-cafd-4e36-ba8d-a7754c96dc78 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:04.569336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fedb53a-cafd-4e36-ba8d-a7754c96dc78 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:04.570345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fedb53a-cafd-4e36-ba8d-a7754c96dc78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:04.570932 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fedb53a-cafd-4e36-ba8d-a7754c96dc78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:04.572843 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fedb53a-cafd-4e36-ba8d-a7754c96dc78 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:04.577483 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a958a-7340-4e56-9082-fe128b868bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:04.578081 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a958a-7340-4e56-9082-fe128b868bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:04.578371 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a958a-7340-4e56-9082-fe128b868bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:04.578649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a958a-7340-4e56-9082-fe128b868bd4 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:04.579327 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a958a-7340-4e56-9082-fe128b868bd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:04.579629 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a958a-7340-4e56-9082-fe128b868bd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:04.581036 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a958a-7340-4e56-9082-fe128b868bd4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:04.592530 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 272/535] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:04.593641 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a958a-7340-4e56-9082-fe128b868bd4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:04.597635 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 264/536] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:04.599910 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553c3e0-da4a-4649-9100-7a0911dbfba0 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:04.600223 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553c3e0-da4a-4649-9100-7a0911dbfba0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:04.600496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553c3e0-da4a-4649-9100-7a0911dbfba0 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:04.605664 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 265/537] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:04.605664 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553c3e0-da4a-4649-9100-7a0911dbfba0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:04.700600 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a958a-7340-4e56-9082-fe128b868bd4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:04.716013 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:04.718222 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:04.721853 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553c3e0-da4a-4649-9100-7a0911dbfba0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:04.735276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b2d23-2ee0-4b07-8df6-4055fb440f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:04.735905 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b2d23-2ee0-4b07-8df6-4055fb440f2c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:04.736179 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b2d23-2ee0-4b07-8df6-4055fb440f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:04.736444 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b2d23-2ee0-4b07-8df6-4055fb440f2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:04.743053 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b2d23-2ee0-4b07-8df6-4055fb440f2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:04.744497 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b2d23-2ee0-4b07-8df6-4055fb440f2c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:04.757518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-693b2d23-2ee0-4b07-8df6-4055fb440f2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:04.757518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:04.757518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:04.757518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:04.757518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:04.757518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:04.757518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:04.757518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:04.757518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:04.757518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:04.757518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:04.757518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:04.757518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:04.757518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:04.757518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:04.764137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:04.764137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:04.764137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:04.764137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:04.764137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:04.764137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:04.764137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:04.764137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:04.764137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:04.764137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:04.764137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:04.764137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:04.764137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:04.764137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:04.764137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:04.764137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:04.764137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:04.765018 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:04.765018 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:04.765018 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:04.765018 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:04.765018 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:04.765018 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:04.765018 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:04.765018 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:04.765430 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3c116-722c-48ec-90f7-34d015335ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:04.765430 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3c116-722c-48ec-90f7-34d015335ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:04.765430 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3c116-722c-48ec-90f7-34d015335ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:04.765430 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3c116-722c-48ec-90f7-34d015335ab8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:04.765687 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 273/538] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:04.765753 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3c116-722c-48ec-90f7-34d015335ab8 None admin] RBAC: Adding query filter params (role.id=303e0fba379a41019ecc2e119d87fb1b, scope.project.id=d11f33af53374dffa61132642a7770f4, user.id=56d5c4afb9844fbbbe699f958deae6e4) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:04.765753 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3c116-722c-48ec-90f7-34d015335ab8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:04.765886 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b2d23-2ee0-4b07-8df6-4055fb440f2c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:04.765961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3c116-722c-48ec-90f7-34d015335ab8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:04.770874 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 266/539] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:55:04 2022] GET /identity/v3/role_assignments?user.id=56d5c4afb9844fbbbe699f958deae6e4&scope.project.id=d11f33af53374dffa61132642a7770f4&role.id=303e0fba379a41019ecc2e119d87fb1b => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:04.772048 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6b90d-328a-49de-b0a4-1236952eb70a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:04.772326 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6b90d-328a-49de-b0a4-1236952eb70a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:04.772593 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6b90d-328a-49de-b0a4-1236952eb70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:04.794776 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1ee6b90d-328a-49de-b0a4-1236952eb70a None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:04.796175 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1ee6b90d-328a-49de-b0a4-1236952eb70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:04.890384 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 267/540] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:55:04.890384 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6b90d-328a-49de-b0a4-1236952eb70a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:04.930183 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b2d23-2ee0-4b07-8df6-4055fb440f2c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:04.983800 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac9849-ef57-428a-a2ba-e03421a65dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:04.984332 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac9849-ef57-428a-a2ba-e03421a65dea None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:04.984865 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac9849-ef57-428a-a2ba-e03421a65dea None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:04.985138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac9849-ef57-428a-a2ba-e03421a65dea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:04.986263 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ac9849-ef57-428a-a2ba-e03421a65dea None admin] RBAC: Adding query filter params (role.id=303e0fba379a41019ecc2e119d87fb1b, scope.project.id=d11f33af53374dffa61132642a7770f4, user.id=e1534cfef9e9428bb2e010fff200bc19) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:04.986602 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ac9849-ef57-428a-a2ba-e03421a65dea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:04.988053 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ac9849-ef57-428a-a2ba-e03421a65dea None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:04.992343 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:05.001212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 274/541] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:55:04 2022] GET /identity/v3/role_assignments?user.id=e1534cfef9e9428bb2e010fff200bc19&scope.project.id=d11f33af53374dffa61132642a7770f4&role.id=303e0fba379a41019ecc2e119d87fb1b => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:05.002531 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac9849-ef57-428a-a2ba-e03421a65dea None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:05.002990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6b90d-328a-49de-b0a4-1236952eb70a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:05.077199 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473c83a-bb94-4b7d-ab84-a58f1575f0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:05.077831 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473c83a-bb94-4b7d-ab84-a58f1575f0bc None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:05.078101 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473c83a-bb94-4b7d-ab84-a58f1575f0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:05.078378 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473c83a-bb94-4b7d-ab84-a58f1575f0bc None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:05.079277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473c83a-bb94-4b7d-ab84-a58f1575f0bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:05.079584 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473c83a-bb94-4b7d-ab84-a58f1575f0bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:05.081068 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473c83a-bb94-4b7d-ab84-a58f1575f0bc None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:05.095532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 268/542] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:05.099723 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:05.102800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473c83a-bb94-4b7d-ab84-a58f1575f0bc None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:05.187102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac9849-ef57-428a-a2ba-e03421a65dea None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:05.226994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3edda8d-faa2-46da-8f88-7619893b5edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:05.227635 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3edda8d-faa2-46da-8f88-7619893b5edc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:05.227912 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3edda8d-faa2-46da-8f88-7619893b5edc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:05.228189 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3edda8d-faa2-46da-8f88-7619893b5edc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:05.237484 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473c83a-bb94-4b7d-ab84-a58f1575f0bc None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:05.241645 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3edda8d-faa2-46da-8f88-7619893b5edc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:05.247721 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c3edda8d-faa2-46da-8f88-7619893b5edc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:05.247721 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:05.247721 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:05.247721 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:05.247721 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:05.247721 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:05.247721 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:05.247721 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:05.247721 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:05.247721 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:05.247721 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:05.247721 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:05.247721 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:05.247721 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:05.247721 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:05.247721 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:05.248820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:05.248820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:05.248820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:05.248820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:05.248820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:05.248820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:05.248820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:05.248820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:05.248820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:05.248820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:05.248820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:05.248820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:05.248820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:05.248820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:05.248820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:05.248820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:05.249802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:05.249802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:05.249802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:05.249802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:05.249802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:05.249802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:05.249802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:05.249802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:05.249802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:05.249802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:05.249802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:05.249802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:05.249802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 275/543] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 10 16:55:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:05.250631 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd51760-7070-43ff-9cdf-5dc016b14062 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:05.250631 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd51760-7070-43ff-9cdf-5dc016b14062 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:05.250631 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd51760-7070-43ff-9cdf-5dc016b14062 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:05.255419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 276/544] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:05.256908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd51760-7070-43ff-9cdf-5dc016b14062 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:05.285687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb8a5e-ad52-47cf-9ad6-b2e5d36922fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:05.286964 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb8a5e-ad52-47cf-9ad6-b2e5d36922fa None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:05.287649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb8a5e-ad52-47cf-9ad6-b2e5d36922fa None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:05.288062 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb8a5e-ad52-47cf-9ad6-b2e5d36922fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:05.289189 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfb8a5e-ad52-47cf-9ad6-b2e5d36922fa None admin] RBAC: Adding query filter params (role.id=303e0fba379a41019ecc2e119d87fb1b, scope.project.id=d11f33af53374dffa61132642a7770f4, user.id=10f4f54890624da29ff217081445b6bf) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:05.290040 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfb8a5e-ad52-47cf-9ad6-b2e5d36922fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:05.292230 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfb8a5e-ad52-47cf-9ad6-b2e5d36922fa None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:05.300920 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 269/545] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:55:05 2022] GET /identity/v3/role_assignments?user.id=10f4f54890624da29ff217081445b6bf&scope.project.id=d11f33af53374dffa61132642a7770f4&role.id=303e0fba379a41019ecc2e119d87fb1b => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:05.308043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6aeb4f-aab5-42bb-a261-32c00f44da6e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:05.308877 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6aeb4f-aab5-42bb-a261-32c00f44da6e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:05.311634 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6aeb4f-aab5-42bb-a261-32c00f44da6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:05.348818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7c6aeb4f-aab5-42bb-a261-32c00f44da6e None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:05.350059 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7c6aeb4f-aab5-42bb-a261-32c00f44da6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:05.365907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:05.438485 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd51760-7070-43ff-9cdf-5dc016b14062 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:05.443945 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:05.462049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 270/546] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:55:05.494235 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112b13c-7c5c-49f4-b9c5-3a930645c703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:05.495252 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112b13c-7c5c-49f4-b9c5-3a930645c703 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:05.496689 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112b13c-7c5c-49f4-b9c5-3a930645c703 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:05.497184 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112b13c-7c5c-49f4-b9c5-3a930645c703 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:05.498089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5112b13c-7c5c-49f4-b9c5-3a930645c703 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:05.498621 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5112b13c-7c5c-49f4-b9c5-3a930645c703 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:05.500235 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5112b13c-7c5c-49f4-b9c5-3a930645c703 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:05.512093 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6aeb4f-aab5-42bb-a261-32c00f44da6e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:05.515027 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 277/547] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 10 16:55:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:05.525748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112b13c-7c5c-49f4-b9c5-3a930645c703 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:05.656761 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112b13c-7c5c-49f4-b9c5-3a930645c703 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:05.704166 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2b2c2-b0a0-40cb-b6b0-6606f13df0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:05.705177 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2b2c2-b0a0-40cb-b6b0-6606f13df0a2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:05.705653 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2b2c2-b0a0-40cb-b6b0-6606f13df0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:05.706116 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2b2c2-b0a0-40cb-b6b0-6606f13df0a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:05.726373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6aeb4f-aab5-42bb-a261-32c00f44da6e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:05.729207 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a2b2c2-b0a0-40cb-b6b0-6606f13df0a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:05.730899 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a2b2c2-b0a0-40cb-b6b0-6606f13df0a2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:05.754161 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3677e35-ec37-4b66-ade5-fd2a0c8e4141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:05.754828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3677e35-ec37-4b66-ade5-fd2a0c8e4141 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:05.755177 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3677e35-ec37-4b66-ade5-fd2a0c8e4141 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:05.755519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3677e35-ec37-4b66-ade5-fd2a0c8e4141 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:05.756764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-97a2b2c2-b0a0-40cb-b6b0-6606f13df0a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:05.756764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:05.756764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:05.756764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:05.756764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:05.756764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:05.756764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:05.756764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:05.756764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:05.756764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:05.756764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:05.756764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:05.756764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:05.756764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:05.756764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:05.756764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:05.757599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:05.757599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:05.757599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:05.757599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:05.757599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:05.757599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:05.757599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:05.757599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:05.757599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:05.757599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:05.757599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:05.757599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:05.757599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:05.757599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:05.757599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:05.757599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:05.757599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:05.758481 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:05.758481 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:05.758481 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:05.763577 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 278/548] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:05.770981 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2b2c2-b0a0-40cb-b6b0-6606f13df0a2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:05.777429 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3677e35-ec37-4b66-ade5-fd2a0c8e4141 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:05.789738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-b3677e35-ec37-4b66-ade5-fd2a0c8e4141 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:05.789738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:05.789738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:05.789738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:05.789738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:05.789738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:05.789738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:05.789738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:05.789738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:05.789738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:05.789738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:05.789738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:05.789738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:05.789738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:05.789738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:05.789738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:05.790856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:05.790856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:05.790856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:05.790856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:05.790856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:05.790856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:05.790856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:05.790856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:05.790856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:05.790856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:05.790856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:05.790856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:05.790856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:05.790856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:05.790856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:05.790856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:05.792612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:05.792612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:05.792612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:05.792612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:05.792612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:05.792612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:05.792612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:05.792612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:05.792612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:05.792612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:05.792612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:05.792612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:05.792612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 271/549] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 10 16:55:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:05.801535 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3677e35-ec37-4b66-ade5-fd2a0c8e4141 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:05.805338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:05.822736 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:05.957802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2b2c2-b0a0-40cb-b6b0-6606f13df0a2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:06.005990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3677e35-ec37-4b66-ade5-fd2a0c8e4141 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:06.006978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b57f9-1f5f-401b-a68a-357e144a795c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:06.009255 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b57f9-1f5f-401b-a68a-357e144a795c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:06.009739 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b57f9-1f5f-401b-a68a-357e144a795c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:06.010202 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b57f9-1f5f-401b-a68a-357e144a795c None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:06.011881 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b57f9-1f5f-401b-a68a-357e144a795c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:06.012386 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b57f9-1f5f-401b-a68a-357e144a795c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:06.016138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b57f9-1f5f-401b-a68a-357e144a795c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:06.036515 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 279/550] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:06.042945 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b57f9-1f5f-401b-a68a-357e144a795c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:06.079376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a3d3d-1722-48b2-b50c-a6072685b633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:06.080334 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a3d3d-1722-48b2-b50c-a6072685b633 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:06.080832 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a3d3d-1722-48b2-b50c-a6072685b633 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:06.082409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a3d3d-1722-48b2-b50c-a6072685b633 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:06.083877 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a3d3d-1722-48b2-b50c-a6072685b633 None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:06.084398 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a3d3d-1722-48b2-b50c-a6072685b633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:06.086944 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a3d3d-1722-48b2-b50c-a6072685b633 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:06.106232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 272/551] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 10 16:55:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:06.112790 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a3d3d-1722-48b2-b50c-a6072685b633 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:06.165959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:06.189344 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b57f9-1f5f-401b-a68a-357e144a795c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:06.225779 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e535c-2368-44a6-895d-664ea247baa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:06.226939 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e535c-2368-44a6-895d-664ea247baa1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:06.227458 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e535c-2368-44a6-895d-664ea247baa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:06.227930 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e535c-2368-44a6-895d-664ea247baa1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:06.236264 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e535c-2368-44a6-895d-664ea247baa1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:06.238144 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e535c-2368-44a6-895d-664ea247baa1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:06.250863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-1c0e535c-2368-44a6-895d-664ea247baa1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:55:06.250863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:06.250863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:06.250863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:06.250863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:06.250863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:06.250863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:06.250863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:06.250863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:06.250863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:06.250863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:06.250863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:06.250863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:06.250863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:06.250863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:06.250863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:06.251806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:06.251806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:06.251806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:06.251806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:06.251806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:06.251806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:06.251806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:06.251806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:06.251806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:06.251806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:06.251806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:06.251806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:06.251806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:06.251806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:06.251806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:06.251806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:06.251806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:06.251806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:06.252716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:06.252716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:06.252716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:06.252716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:06.252716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:06.252716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:06.252716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:06.252716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:06.252716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:55:06.252716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:06.255380 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 280/552] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 10 16:55:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:06.261924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e535c-2368-44a6-895d-664ea247baa1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:06.276509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:06.377464 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e535c-2368-44a6-895d-664ea247baa1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:06.408373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a3d3d-1722-48b2-b50c-a6072685b633 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:06.415391 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d680e6-a89c-4faa-ac38-3263ee0a44b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:06.416257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d680e6-a89c-4faa-ac38-3263ee0a44b8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:06.416824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d680e6-a89c-4faa-ac38-3263ee0a44b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:06.418081 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d680e6-a89c-4faa-ac38-3263ee0a44b8 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:06.419622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d680e6-a89c-4faa-ac38-3263ee0a44b8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:06.420130 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d680e6-a89c-4faa-ac38-3263ee0a44b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:06.421810 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d680e6-a89c-4faa-ac38-3263ee0a44b8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:06.437329 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:06.441513 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 281/553] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 10 16:55:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:06.447463 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d680e6-a89c-4faa-ac38-3263ee0a44b8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:06.491074 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ec049-6eac-463b-9959-4eee3a954515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:06.491716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ec049-6eac-463b-9959-4eee3a954515 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:06.491996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ec049-6eac-463b-9959-4eee3a954515 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:06.492276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ec049-6eac-463b-9959-4eee3a954515 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:06.509971 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ec049-6eac-463b-9959-4eee3a954515 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:06.511527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ec049-6eac-463b-9959-4eee3a954515 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:06.540946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-ec1ec049-6eac-463b-9959-4eee3a954515 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:06.540946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:06.540946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:06.540946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:06.540946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:06.540946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:06.540946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:06.540946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:06.540946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:06.540946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:06.540946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:06.540946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:06.540946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:06.540946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:06.540946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:06.540946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:06.541907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:06.541907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:06.541907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:06.541907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:06.541907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:06.541907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:06.541907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:06.541907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:06.541907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:06.541907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:06.541907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:06.541907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:06.541907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:06.541907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:06.541907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:06.541907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:06.541907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:06.543672 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:06.543672 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:06.543672 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:06.543672 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 273/554] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 430 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:06.548388 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ec049-6eac-463b-9959-4eee3a954515 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:06.558257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d680e6-a89c-4faa-ac38-3263ee0a44b8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:06.629574 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeac9d6-0e32-4803-9f67-763b7bd91f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:06.630338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeac9d6-0e32-4803-9f67-763b7bd91f88 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:06.630633 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeac9d6-0e32-4803-9f67-763b7bd91f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:06.630901 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeac9d6-0e32-4803-9f67-763b7bd91f88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:06.655364 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeac9d6-0e32-4803-9f67-763b7bd91f88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:06.657302 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeac9d6-0e32-4803-9f67-763b7bd91f88 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:06.675379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-fbeac9d6-0e32-4803-9f67-763b7bd91f88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:06.675379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:06.675379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:06.675379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:06.675379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:06.675379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:06.675379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:06.675379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:06.675379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:06.675379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:06.675379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:06.675379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:06.675379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:06.675379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:06.675379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:06.675379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:06.676535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:06.676535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:06.676535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:06.676535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:06.676535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:06.676535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:06.676535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:06.676535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:06.676535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:06.676535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:06.676535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:06.676535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:06.676535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:06.676535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:06.676535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:06.676535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:06.676535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:06.678006 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:06.678006 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:06.678006 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:06.679980 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 282/555] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:06.685659 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeac9d6-0e32-4803-9f67-763b7bd91f88 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:06.720279 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ec049-6eac-463b-9959-4eee3a954515 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:06.763112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:06.778731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:06.791340 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82962a88-bf18-4b2f-bb3f-627fb47ccd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:06.792193 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82962a88-bf18-4b2f-bb3f-627fb47ccd42 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:06.793178 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82962a88-bf18-4b2f-bb3f-627fb47ccd42 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:06.793647 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82962a88-bf18-4b2f-bb3f-627fb47ccd42 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:06.794560 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82962a88-bf18-4b2f-bb3f-627fb47ccd42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:06.795004 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82962a88-bf18-4b2f-bb3f-627fb47ccd42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:06.796509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82962a88-bf18-4b2f-bb3f-627fb47ccd42 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:06.810935 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 274/556] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:06.816423 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82962a88-bf18-4b2f-bb3f-627fb47ccd42 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:06.888101 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeac9d6-0e32-4803-9f67-763b7bd91f88 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:06.923223 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82962a88-bf18-4b2f-bb3f-627fb47ccd42 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:06.950112 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff02a1-a8a1-4f95-b3fe-14ad07a23f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:06.950801 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff02a1-a8a1-4f95-b3fe-14ad07a23f21 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:06.951179 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff02a1-a8a1-4f95-b3fe-14ad07a23f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:06.951370 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff02a1-a8a1-4f95-b3fe-14ad07a23f21 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:06.951978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff02a1-a8a1-4f95-b3fe-14ad07a23f21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:06.952274 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff02a1-a8a1-4f95-b3fe-14ad07a23f21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:06.953638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff02a1-a8a1-4f95-b3fe-14ad07a23f21 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:06.963103 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36845ee-3a30-4ff7-a69c-249abd9946b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:06.964006 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36845ee-3a30-4ff7-a69c-249abd9946b7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:06.965171 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36845ee-3a30-4ff7-a69c-249abd9946b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:06.965647 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36845ee-3a30-4ff7-a69c-249abd9946b7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:06.973456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36845ee-3a30-4ff7-a69c-249abd9946b7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:06.975221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36845ee-3a30-4ff7-a69c-249abd9946b7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:06.987192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-c36845ee-3a30-4ff7-a69c-249abd9946b7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 16:55:06.987192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:06.987192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:06.987192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:06.987192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:06.987192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:06.987192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:06.987192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:06.987192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:06.987192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:06.987192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:06.987192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:06.987192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:06.987192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:06.987192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:06.987192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:06.988225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:06.988225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:06.988225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:06.988225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:06.988225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:06.988225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:06.988225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:06.988225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:06.988225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:06.988225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:06.988225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:06.988225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:06.988225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:06.988225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:06.988225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:06.988225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:06.988225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:06.988225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:06.989165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:06.989165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:06.989165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:06.989165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:06.989165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:06.989165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:06.989165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:06.989165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:06.989165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 16:55:06.989165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:06.989723 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 283/557] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:06.991803 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 275/558] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:55:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:06.993551 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff02a1-a8a1-4f95-b3fe-14ad07a23f21 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:06.998123 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36845ee-3a30-4ff7-a69c-249abd9946b7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:07.082716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:07.190215 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff02a1-a8a1-4f95-b3fe-14ad07a23f21 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:07.202118 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:07.212560 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36845ee-3a30-4ff7-a69c-249abd9946b7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:07.230105 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74294e30-58da-4d16-b30d-a9a3208eb86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:07.231703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74294e30-58da-4d16-b30d-a9a3208eb86b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:07.232292 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74294e30-58da-4d16-b30d-a9a3208eb86b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:07.232746 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74294e30-58da-4d16-b30d-a9a3208eb86b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:07.242024 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74294e30-58da-4d16-b30d-a9a3208eb86b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:07.244210 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74294e30-58da-4d16-b30d-a9a3208eb86b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:07.258891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87083348-7ee2-4aa0-8a42-36eac930eb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:07.259706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-74294e30-58da-4d16-b30d-a9a3208eb86b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:07.259706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:07.259706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:07.259706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:07.259706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:07.259706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:07.259706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:07.259706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:07.259706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:07.259706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:07.259706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:07.259706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:07.259706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:07.259706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:07.259706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:07.260727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:07.260727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:07.260727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:07.260727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:07.260727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:07.260727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:07.260727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:07.260727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:07.260727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:07.260727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:07.260727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:07.260727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:07.260727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:07.260727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:07.260727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:07.260727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:07.260727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:07.262594 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:07.262594 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:07.262594 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:07.262594 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:07.262594 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:07.262594 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:07.262594 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:07.262594 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:07.263261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87083348-7ee2-4aa0-8a42-36eac930eb07 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:07.263682 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87083348-7ee2-4aa0-8a42-36eac930eb07 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:07.264412 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 284/559] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:07.264875 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87083348-7ee2-4aa0-8a42-36eac930eb07 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:07.265816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87083348-7ee2-4aa0-8a42-36eac930eb07 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:07.266150 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2820f-f25d-4eb5-8f35-cc32fc5659d7 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:07.266473 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87083348-7ee2-4aa0-8a42-36eac930eb07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:07.266647 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2820f-f25d-4eb5-8f35-cc32fc5659d7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:07.267023 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2820f-f25d-4eb5-8f35-cc32fc5659d7 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:07.268353 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87083348-7ee2-4aa0-8a42-36eac930eb07 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:07.268790 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 285/560] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:07.270060 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2820f-f25d-4eb5-8f35-cc32fc5659d7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:07.295068 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 276/561] 10.222.0.45 () {56 vars in 1135 bytes} [Wed Aug 10 16:55:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:07.297385 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3502d-2f55-4fc6-8c0e-532f6f0b9520 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:07.298076 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3502d-2f55-4fc6-8c0e-532f6f0b9520 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:07.298524 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3502d-2f55-4fc6-8c0e-532f6f0b9520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:07.320168 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a2e3502d-2f55-4fc6-8c0e-532f6f0b9520 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:07.321515 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a2e3502d-2f55-4fc6-8c0e-532f6f0b9520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:07.382417 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2820f-f25d-4eb5-8f35-cc32fc5659d7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:07.390263 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 277/562] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:55:07.391365 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae166d7-81b6-43bd-a557-d910a9698efe None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:07.391643 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae166d7-81b6-43bd-a557-d910a9698efe None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:07.391912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae166d7-81b6-43bd-a557-d910a9698efe None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:07.394478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 278/563] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:07.394478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae166d7-81b6-43bd-a557-d910a9698efe None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:07.423932 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:07.436193 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d867a579-e655-4241-bc83-131566207c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:07.438046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867a579-e655-4241-bc83-131566207c56 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:07.438046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867a579-e655-4241-bc83-131566207c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:07.438046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867a579-e655-4241-bc83-131566207c56 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:07.438318 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d867a579-e655-4241-bc83-131566207c56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:07.438483 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d867a579-e655-4241-bc83-131566207c56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:07.439860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d867a579-e655-4241-bc83-131566207c56 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:07.452304 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 286/564] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:07.453570 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb46b4-c9ba-43d5-b490-5671565f6257 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:07.453856 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb46b4-c9ba-43d5-b490-5671565f6257 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:07.454126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb46b4-c9ba-43d5-b490-5671565f6257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:07.471690 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d8cb46b4-c9ba-43d5-b490-5671565f6257 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:07.472862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d8cb46b4-c9ba-43d5-b490-5671565f6257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:07.493484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:07.508991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae166d7-81b6-43bd-a557-d910a9698efe None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:07.541047 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 287/565] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:55:07.548726 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798fce8-20fa-4d90-adff-03c3be308166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:07.549336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798fce8-20fa-4d90-adff-03c3be308166 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:07.549651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798fce8-20fa-4d90-adff-03c3be308166 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:07.549914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798fce8-20fa-4d90-adff-03c3be308166 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:07.555979 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb46b4-c9ba-43d5-b490-5671565f6257 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:07.626333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798fce8-20fa-4d90-adff-03c3be308166 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:07.627805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798fce8-20fa-4d90-adff-03c3be308166 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:07.658932 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:07.677443 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb46b4-c9ba-43d5-b490-5671565f6257 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:07.712594 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42293707-c82f-4c20-aba5-a60168988a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:07.713215 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42293707-c82f-4c20-aba5-a60168988a96 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:07.713490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42293707-c82f-4c20-aba5-a60168988a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:07.713767 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42293707-c82f-4c20-aba5-a60168988a96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:07.724343 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42293707-c82f-4c20-aba5-a60168988a96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:07.729109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-42293707-c82f-4c20-aba5-a60168988a96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:07.729109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:07.729109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:07.729109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:07.729109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:07.729109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:07.729109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:07.729109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:07.729109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:07.729109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:07.729109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:07.729109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:07.729109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:07.729109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:07.729109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:07.729109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:07.730703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:07.730703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:07.730703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:07.730703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:07.730703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:07.730703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:07.730703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:07.730703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:07.730703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:07.730703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:07.730703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:07.730703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:07.730703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:07.730703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:07.730703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:07.730703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:07.731572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:07.731572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:07.731572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:07.731572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:07.731572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:07.731572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:07.731572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:07.731572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:07.731572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:07.731572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:07.731572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:07.731572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:07.732225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 279/566] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:55:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:55:07.732301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 288/567] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 10 16:55:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:07.732301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42293707-c82f-4c20-aba5-a60168988a96 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:07.732775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798fce8-20fa-4d90-adff-03c3be308166 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:07.752052 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:07.832978 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798fce8-20fa-4d90-adff-03c3be308166 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:07.833547 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42293707-c82f-4c20-aba5-a60168988a96 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:07.858584 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:07.865733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8d177-ca64-4974-86a7-a4fbe803952e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:07.866194 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6a22a7-1f98-4690-b1c9-7805f08ff934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:07.866409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8d177-ca64-4974-86a7-a4fbe803952e None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:07.866685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8d177-ca64-4974-86a7-a4fbe803952e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:07.866802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a22a7-1f98-4690-b1c9-7805f08ff934 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:07.872177 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8d177-ca64-4974-86a7-a4fbe803952e None admin] PATH_INFO: `/v3/projects/d11f33af53374dffa61132642a7770f4/users/615cc36e21cc493b823e6cee695ac308/roles/303e0fba379a41019ecc2e119d87fb1b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:07.872326 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a22a7-1f98-4690-b1c9-7805f08ff934 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:07.872326 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a22a7-1f98-4690-b1c9-7805f08ff934 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:07.878036 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6a22a7-1f98-4690-b1c9-7805f08ff934 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:07.881508 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8d177-ca64-4974-86a7-a4fbe803952e None admin] RBAC: Authorizing `identity:create_grant(project_id=d11f33af53374dffa61132642a7770f4, user_id=615cc36e21cc493b823e6cee695ac308, role_id=303e0fba379a41019ecc2e119d87fb1b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:07.882675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ae6a22a7-1f98-4690-b1c9-7805f08ff934 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:07.882675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:07.882675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:07.882675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:07.882675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:07.882675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:07.882675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:07.882675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:07.882675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:07.882675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:07.882675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:07.882675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:07.882675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:07.882675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:07.882675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:07.882675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:07.883785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:07.883785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:07.883785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:07.883785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:07.883785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:07.883785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:07.883785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:07.883785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:07.883785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:07.883785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:07.883785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:07.883785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:07.883785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:07.883785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:07.883785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:07.883785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:07.884734 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:07.884734 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:07.884734 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:07.884734 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:07.884734 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:07.884734 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:07.884734 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:07.884734 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:07.884734 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:07.884734 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:07.884734 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:07.884734 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:07.885444 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8d177-ca64-4974-86a7-a4fbe803952e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:07.885542 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 289/568] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 10 16:55:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:07.885542 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0e4b0-fcb2-45fe-9538-2d85b33bdcfd None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:07.885542 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0e4b0-fcb2-45fe-9538-2d85b33bdcfd None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:07.885744 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0e4b0-fcb2-45fe-9538-2d85b33bdcfd None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:07.887382 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 290/569] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:07.888069 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0e4b0-fcb2-45fe-9538-2d85b33bdcfd None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:07.964063 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:07.986410 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0e4b0-fcb2-45fe-9538-2d85b33bdcfd None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:07.990516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 280/570] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 10 16:55:07 2022] PUT /identity/v3/projects/d11f33af53374dffa61132642a7770f4/users/615cc36e21cc493b823e6cee695ac308/roles/303e0fba379a41019ecc2e119d87fb1b => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:07.995840 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8d177-ca64-4974-86a7-a4fbe803952e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:08.025427 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64bbb9-1403-43c3-833d-cdbf060d4130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:08.026093 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64bbb9-1403-43c3-833d-cdbf060d4130 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:08.026366 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64bbb9-1403-43c3-833d-cdbf060d4130 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:08.026657 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64bbb9-1403-43c3-833d-cdbf060d4130 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:08.039002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d64bbb9-1403-43c3-833d-cdbf060d4130 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:08.040565 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d64bbb9-1403-43c3-833d-cdbf060d4130 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:08.057085 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-6d64bbb9-1403-43c3-833d-cdbf060d4130 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:08.057085 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:08.057085 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:08.057085 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:08.057085 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:08.057085 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:08.057085 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:08.057085 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:08.057085 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:08.057085 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:08.057085 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:08.057085 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:08.057085 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:08.057085 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:08.057085 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:08.057085 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:08.066187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:08.066187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:08.066187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:08.066187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:08.066187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:08.066187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:08.066187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:08.066187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:08.066187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:08.066187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:08.066187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:08.066187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:08.066187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:08.066187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:08.066187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:08.066187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:08.066187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:08.067144 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:08.067144 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:08.067144 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:08.067144 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 291/571] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:08.067144 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64bbb9-1403-43c3-833d-cdbf060d4130 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:08.116403 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8d177-ca64-4974-86a7-a4fbe803952e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:08.162930 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a79f7-3678-4742-bbf8-c741db6005e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:08.163591 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a79f7-3678-4742-bbf8-c741db6005e1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:08.163871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a79f7-3678-4742-bbf8-c741db6005e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:08.164140 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a79f7-3678-4742-bbf8-c741db6005e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:08.164807 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a79f7-3678-4742-bbf8-c741db6005e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:08.165100 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a79f7-3678-4742-bbf8-c741db6005e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:08.166487 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a79f7-3678-4742-bbf8-c741db6005e1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:08.175421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 281/572] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 10 16:55:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:08.176629 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d99cd-4501-4dcd-a5a4-b9f86ad2a73a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:08.176910 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d99cd-4501-4dcd-a5a4-b9f86ad2a73a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:08.177178 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d99cd-4501-4dcd-a5a4-b9f86ad2a73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:08.182315 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64bbb9-1403-43c3-833d-cdbf060d4130 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:08.197676 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b11d99cd-4501-4dcd-a5a4-b9f86ad2a73a None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:08.199777 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b11d99cd-4501-4dcd-a5a4-b9f86ad2a73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:08.211154 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:08.218983 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260eb316-c376-4526-b7eb-276158fe9d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:08.219522 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260eb316-c376-4526-b7eb-276158fe9d20 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:08.219805 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260eb316-c376-4526-b7eb-276158fe9d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:08.220078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260eb316-c376-4526-b7eb-276158fe9d20 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:08.220738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260eb316-c376-4526-b7eb-276158fe9d20 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:08.221035 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260eb316-c376-4526-b7eb-276158fe9d20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:08.222415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260eb316-c376-4526-b7eb-276158fe9d20 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:08.236521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 292/573] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 10 16:55:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:08.236521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260eb316-c376-4526-b7eb-276158fe9d20 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:08.239876 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:08.275901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 282/574] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:55:08.277614 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d99cd-4501-4dcd-a5a4-b9f86ad2a73a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:08.367118 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260eb316-c376-4526-b7eb-276158fe9d20 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:08.391819 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d99cd-4501-4dcd-a5a4-b9f86ad2a73a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:08.409267 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4692f0d-b077-4acd-adeb-4a843b7ef695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:08.409985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4692f0d-b077-4acd-adeb-4a843b7ef695 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:08.410263 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4692f0d-b077-4acd-adeb-4a843b7ef695 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:08.410558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4692f0d-b077-4acd-adeb-4a843b7ef695 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:08.411205 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4692f0d-b077-4acd-adeb-4a843b7ef695 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:08.411502 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4692f0d-b077-4acd-adeb-4a843b7ef695 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:08.412910 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4692f0d-b077-4acd-adeb-4a843b7ef695 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:08.429304 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 293/575] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:08.429304 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4692f0d-b077-4acd-adeb-4a843b7ef695 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:08.436384 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adbb6e-36a9-49fe-bbc5-84ffb0ca3123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:08.437177 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adbb6e-36a9-49fe-bbc5-84ffb0ca3123 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:08.437494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adbb6e-36a9-49fe-bbc5-84ffb0ca3123 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:08.437766 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adbb6e-36a9-49fe-bbc5-84ffb0ca3123 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:08.451157 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23adbb6e-36a9-49fe-bbc5-84ffb0ca3123 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:08.452616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23adbb6e-36a9-49fe-bbc5-84ffb0ca3123 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:08.468252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-23adbb6e-36a9-49fe-bbc5-84ffb0ca3123 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:08.468252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:08.468252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:08.468252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:08.468252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:08.468252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:08.468252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:08.468252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:08.468252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:08.468252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:08.468252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:08.468252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:08.468252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:08.468252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:08.468252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:08.468252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:08.469282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:08.469282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:08.469282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:08.469282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:08.469282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:08.469282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:08.469282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:08.469282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:08.469282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:08.469282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:08.469282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:08.469282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:08.469282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:08.469282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:08.469282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:08.469282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:08.469282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:08.470165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:08.470165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:08.470165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:08.470165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 283/576] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:08.470165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adbb6e-36a9-49fe-bbc5-84ffb0ca3123 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:08.485791 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:08.497464 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:08.544535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4692f0d-b077-4acd-adeb-4a843b7ef695 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:08.582111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adbb6e-36a9-49fe-bbc5-84ffb0ca3123 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:08.584326 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee41e62-f7ec-40ed-af14-8bd3a84eb13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:08.584961 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee41e62-f7ec-40ed-af14-8bd3a84eb13d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:08.585240 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee41e62-f7ec-40ed-af14-8bd3a84eb13d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:08.585513 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee41e62-f7ec-40ed-af14-8bd3a84eb13d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:08.596850 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee41e62-f7ec-40ed-af14-8bd3a84eb13d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:08.598379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee41e62-f7ec-40ed-af14-8bd3a84eb13d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:08.616248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-8ee41e62-f7ec-40ed-af14-8bd3a84eb13d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:08.616248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:08.616248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:08.616248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:08.616248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:08.616248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:08.616248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:08.616248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:08.616248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:08.616248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:08.616248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:08.616248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:08.616248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:08.616248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:08.616248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:08.616248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:08.617269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:08.617269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:08.617269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:08.617269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:08.617269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:08.617269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:08.617269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:08.617269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:08.617269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:08.617269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:08.617269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:08.617269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:08.617269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:08.617269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:08.617269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:08.617269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:08.617269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:08.618174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:08.618174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:08.618174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:08.618174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 294/577] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:08.619166 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee41e62-f7ec-40ed-af14-8bd3a84eb13d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:08.637093 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea34f22-cfcc-4af6-8156-3980bb68959a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:08.637875 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea34f22-cfcc-4af6-8156-3980bb68959a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:08.638261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea34f22-cfcc-4af6-8156-3980bb68959a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:08.638510 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea34f22-cfcc-4af6-8156-3980bb68959a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:08.652589 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea34f22-cfcc-4af6-8156-3980bb68959a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:08.659159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-cea34f22-cfcc-4af6-8156-3980bb68959a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:08.659159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:08.659159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:08.659159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:08.659159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:08.659159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:08.659159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:08.659159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:08.659159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:08.659159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:08.659159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:08.659159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:08.659159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:08.659159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:08.659159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:08.659159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:08.660136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:08.660136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:08.660136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:08.660136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:08.660136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:08.660136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:08.660136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:08.660136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:08.660136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:08.660136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:08.660136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:08.660136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:08.660136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:08.660136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:08.660136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:08.660136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:08.666538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:08.666538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:08.666538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:08.666538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:08.666538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:08.666538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:08.666538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:08.666538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:08.666538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:08.666538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:08.666538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:08.666538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:08.666538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 284/578] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 10 16:55:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:08.666538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea34f22-cfcc-4af6-8156-3980bb68959a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:08.733015 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee41e62-f7ec-40ed-af14-8bd3a84eb13d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:08.734651 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:08.752737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:08.768533 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90361e99-e1c7-4b56-b864-4b4bd5a62870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:08.769167 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90361e99-e1c7-4b56-b864-4b4bd5a62870 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:08.769445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90361e99-e1c7-4b56-b864-4b4bd5a62870 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:08.769722 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90361e99-e1c7-4b56-b864-4b4bd5a62870 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:08.771922 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea34f22-cfcc-4af6-8156-3980bb68959a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:08.776525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90361e99-e1c7-4b56-b864-4b4bd5a62870 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:08.778106 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90361e99-e1c7-4b56-b864-4b4bd5a62870 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:08.790806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-90361e99-e1c7-4b56-b864-4b4bd5a62870 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:08.790806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:08.790806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:08.790806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:08.790806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:08.790806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:08.790806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:08.790806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:08.790806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:08.790806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:08.790806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:08.790806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:08.790806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:08.790806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:08.790806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:08.791737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:08.791737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:08.791737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:08.791737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:08.791737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:08.791737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:08.791737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:08.791737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:08.791737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:08.791737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:08.791737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:08.791737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:08.791737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:08.791737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:08.791737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:08.791737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:08.791737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:08.792960 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:08.792960 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:08.792960 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:08.792960 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:08.792960 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:08.792960 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:08.792960 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:08.792960 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:08.796792 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 295/579] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:08.797642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90361e99-e1c7-4b56-b864-4b4bd5a62870 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:08.813809 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536146ec-2036-4fea-bbb0-fac2b924fea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:08.815137 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536146ec-2036-4fea-bbb0-fac2b924fea3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:08.815931 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536146ec-2036-4fea-bbb0-fac2b924fea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:08.816369 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536146ec-2036-4fea-bbb0-fac2b924fea3 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:08.817416 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536146ec-2036-4fea-bbb0-fac2b924fea3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:08.818252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536146ec-2036-4fea-bbb0-fac2b924fea3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:08.819807 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536146ec-2036-4fea-bbb0-fac2b924fea3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:08.837210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 285/580] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:08.838086 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536146ec-2036-4fea-bbb0-fac2b924fea3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:08.904291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90361e99-e1c7-4b56-b864-4b4bd5a62870 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:08.939505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c6223-369b-4999-a88a-e10161a4ba03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:08.940106 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c6223-369b-4999-a88a-e10161a4ba03 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:08.940382 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c6223-369b-4999-a88a-e10161a4ba03 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:08.940651 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c6223-369b-4999-a88a-e10161a4ba03 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:08.941282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c6223-369b-4999-a88a-e10161a4ba03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:08.941580 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c6223-369b-4999-a88a-e10161a4ba03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:08.942983 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c6223-369b-4999-a88a-e10161a4ba03 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:08.946656 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536146ec-2036-4fea-bbb0-fac2b924fea3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:08.955167 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 296/581] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:08.956040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c6223-369b-4999-a88a-e10161a4ba03 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:08.985435 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:08.988173 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca83d0e-1bd8-4096-9f7b-4cb2ff9b780c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:08.988772 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca83d0e-1bd8-4096-9f7b-4cb2ff9b780c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:08.989070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca83d0e-1bd8-4096-9f7b-4cb2ff9b780c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:08.989339 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca83d0e-1bd8-4096-9f7b-4cb2ff9b780c None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:08.989999 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca83d0e-1bd8-4096-9f7b-4cb2ff9b780c None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:08.990294 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca83d0e-1bd8-4096-9f7b-4cb2ff9b780c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:08.991727 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca83d0e-1bd8-4096-9f7b-4cb2ff9b780c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:09.001624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 286/582] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 10 16:55:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:09.002486 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca83d0e-1bd8-4096-9f7b-4cb2ff9b780c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:09.008280 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:09.072490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c6223-369b-4999-a88a-e10161a4ba03 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:09.110562 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603aaca6-8a78-48c1-8e28-8ca80374f0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:09.111184 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603aaca6-8a78-48c1-8e28-8ca80374f0af None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:09.111458 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603aaca6-8a78-48c1-8e28-8ca80374f0af None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:09.111727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603aaca6-8a78-48c1-8e28-8ca80374f0af None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:09.112436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603aaca6-8a78-48c1-8e28-8ca80374f0af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:09.112740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603aaca6-8a78-48c1-8e28-8ca80374f0af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:09.112937 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca83d0e-1bd8-4096-9f7b-4cb2ff9b780c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:09.114195 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603aaca6-8a78-48c1-8e28-8ca80374f0af None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:09.127612 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 297/583] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:09.128624 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603aaca6-8a78-48c1-8e28-8ca80374f0af None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:09.161268 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471e027-0bea-4f87-96b8-d97ee339ad9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:09.161881 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471e027-0bea-4f87-96b8-d97ee339ad9f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:09.162152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471e027-0bea-4f87-96b8-d97ee339ad9f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:09.162445 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471e027-0bea-4f87-96b8-d97ee339ad9f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:09.170987 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b471e027-0bea-4f87-96b8-d97ee339ad9f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:09.172419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b471e027-0bea-4f87-96b8-d97ee339ad9f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:09.187094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-b471e027-0bea-4f87-96b8-d97ee339ad9f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 16:55:09.187094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:09.187094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:09.187094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:09.187094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:09.187094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:09.187094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:09.187094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:09.187094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:09.187094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:09.187094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:09.187094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:09.187094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:09.187094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:09.187094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:09.187094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:09.188117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:09.188117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:09.188117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:09.188117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:09.188117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:09.188117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:09.188117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:09.188117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:09.188117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:09.188117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:09.188117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:09.188117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:09.188117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:09.188117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:09.188117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:09.188117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:09.188117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:09.188117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:09.189111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:09.189111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:09.189111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:09.189111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:09.189111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:09.189111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:09.189111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:09.189111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:09.189111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 16:55:09.189111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:09.189111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 287/584] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:55:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:09.189111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471e027-0bea-4f87-96b8-d97ee339ad9f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:09.230628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:09.249982 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603aaca6-8a78-48c1-8e28-8ca80374f0af None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:09.254819 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:09.283308 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4721d07e-21c6-430b-9ce9-5386c249ab83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:09.284003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721d07e-21c6-430b-9ce9-5386c249ab83 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:09.284288 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721d07e-21c6-430b-9ce9-5386c249ab83 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:09.284562 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721d07e-21c6-430b-9ce9-5386c249ab83 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:09.293263 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4721d07e-21c6-430b-9ce9-5386c249ab83 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:09.294738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4721d07e-21c6-430b-9ce9-5386c249ab83 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:09.297416 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471e027-0bea-4f87-96b8-d97ee339ad9f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:09.305510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4721d07e-21c6-430b-9ce9-5386c249ab83 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 16:55:09.305510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:09.305510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:09.305510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:09.305510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:09.305510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:09.305510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:09.305510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:09.305510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:09.305510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:09.305510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:09.305510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:09.305510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:09.305510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:09.305510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:09.305510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:09.306638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:09.306638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:09.306638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:09.306638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:09.306638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:09.306638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:09.306638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:09.306638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:09.306638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:09.306638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:09.306638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:09.306638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:09.306638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:09.306638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:09.306638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:09.306638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:09.306638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:09.306638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:09.307575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:09.307575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:09.307575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:09.307575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:09.307575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:09.307575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:09.307575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:09.307575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:09.307575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 16:55:09.307575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:09.310671 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 298/585] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 10 16:55:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:09.311962 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4721d07e-21c6-430b-9ce9-5386c249ab83 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:09.334806 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882fab03-4334-45c3-827e-5d7ca6175056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:09.335502 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fab03-4334-45c3-827e-5d7ca6175056 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:09.335785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fab03-4334-45c3-827e-5d7ca6175056 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:09.336056 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fab03-4334-45c3-827e-5d7ca6175056 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:09.347241 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882fab03-4334-45c3-827e-5d7ca6175056 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:09.348677 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882fab03-4334-45c3-827e-5d7ca6175056 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:09.363636 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-882fab03-4334-45c3-827e-5d7ca6175056 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:09.363636 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:09.363636 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:09.363636 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:09.363636 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:09.363636 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:09.363636 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:09.363636 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:09.363636 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:09.363636 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:09.363636 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:09.363636 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:09.363636 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:09.363636 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:09.363636 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:09.363636 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:09.364602 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:09.364602 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:09.364602 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:09.364602 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:09.364602 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:09.364602 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:09.364602 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:09.364602 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:09.364602 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:09.364602 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:09.364602 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:09.364602 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:09.364602 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:09.364602 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:09.364602 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:09.364602 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:09.364602 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:09.365557 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:09.365557 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:09.365557 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:09.365557 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 288/586] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:09.366575 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882fab03-4334-45c3-827e-5d7ca6175056 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:09.438478 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4721d07e-21c6-430b-9ce9-5386c249ab83 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:09.473940 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a5a11-8c18-4045-8ad9-eb8564044446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:09.474570 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a5a11-8c18-4045-8ad9-eb8564044446 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:09.474852 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a5a11-8c18-4045-8ad9-eb8564044446 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:09.475121 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a5a11-8c18-4045-8ad9-eb8564044446 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:09.475843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951a5a11-8c18-4045-8ad9-eb8564044446 None admin] RBAC: Adding query filter params (role.id=303e0fba379a41019ecc2e119d87fb1b, scope.project.id=d11f33af53374dffa61132642a7770f4, user.id=4986821ef9f84b99a835e387635aa9a8) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:09.476149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951a5a11-8c18-4045-8ad9-eb8564044446 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:09.476884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882fab03-4334-45c3-827e-5d7ca6175056 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:09.477595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951a5a11-8c18-4045-8ad9-eb8564044446 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:09.490708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 299/587] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:55:09 2022] GET /identity/v3/role_assignments?user.id=4986821ef9f84b99a835e387635aa9a8&scope.project.id=d11f33af53374dffa61132642a7770f4&role.id=303e0fba379a41019ecc2e119d87fb1b => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:09.490708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a5a11-8c18-4045-8ad9-eb8564044446 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:09.501188 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:09.504276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:09.530720 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183fdb0-2fd0-421d-8378-ff5a3909dcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:09.531372 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183fdb0-2fd0-421d-8378-ff5a3909dcfe None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:09.531650 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183fdb0-2fd0-421d-8378-ff5a3909dcfe None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:09.531986 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183fdb0-2fd0-421d-8378-ff5a3909dcfe None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:09.533721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183fdb0-2fd0-421d-8378-ff5a3909dcfe None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:09.534073 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183fdb0-2fd0-421d-8378-ff5a3909dcfe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:09.536352 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183fdb0-2fd0-421d-8378-ff5a3909dcfe None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:09.567005 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 289/588] 10.222.0.45 () {56 vars in 1137 bytes} [Wed Aug 10 16:55:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:09.568064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183fdb0-2fd0-421d-8378-ff5a3909dcfe None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:09.639812 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a5a11-8c18-4045-8ad9-eb8564044446 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:09.687609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84fb20-930a-47d8-afb3-b9db030872f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:09.688320 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84fb20-930a-47d8-afb3-b9db030872f7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:09.688602 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84fb20-930a-47d8-afb3-b9db030872f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:09.688871 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84fb20-930a-47d8-afb3-b9db030872f7 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:09.689714 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf84fb20-930a-47d8-afb3-b9db030872f7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:09.690021 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf84fb20-930a-47d8-afb3-b9db030872f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:09.691541 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf84fb20-930a-47d8-afb3-b9db030872f7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:09.709252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183fdb0-2fd0-421d-8378-ff5a3909dcfe None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:09.717098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 300/589] 10.222.0.45 () {56 vars in 1139 bytes} [Wed Aug 10 16:55:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:09.717098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84fb20-930a-47d8-afb3-b9db030872f7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:09.750381 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5652a2-caf3-4584-a8df-67241022851c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:09.751409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5652a2-caf3-4584-a8df-67241022851c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:09.751893 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5652a2-caf3-4584-a8df-67241022851c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:09.752388 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5652a2-caf3-4584-a8df-67241022851c None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:09.753238 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5652a2-caf3-4584-a8df-67241022851c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:09.753760 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5652a2-caf3-4584-a8df-67241022851c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:09.755458 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5652a2-caf3-4584-a8df-67241022851c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:09.772897 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 290/590] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:09.774114 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5652a2-caf3-4584-a8df-67241022851c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:09.785389 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:09.810298 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:09.826663 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84fb20-930a-47d8-afb3-b9db030872f7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:09.864335 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e85f28-de65-4b9d-b9c4-ebbe98f12de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:09.864964 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e85f28-de65-4b9d-b9c4-ebbe98f12de8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:09.865242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e85f28-de65-4b9d-b9c4-ebbe98f12de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:09.865506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e85f28-de65-4b9d-b9c4-ebbe98f12de8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:09.879013 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e85f28-de65-4b9d-b9c4-ebbe98f12de8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:09.880263 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e85f28-de65-4b9d-b9c4-ebbe98f12de8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:09.894638 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5652a2-caf3-4584-a8df-67241022851c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:09.896071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-f3e85f28-de65-4b9d-b9c4-ebbe98f12de8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:09.896071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:09.896071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:09.896071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:09.896071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:09.896071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:09.896071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:09.896071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:09.896071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:09.896071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:09.896071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:09.896071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:09.896071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:09.896071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:09.896071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:09.896071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:09.896985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:09.896985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:09.896985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:09.896985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:09.896985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:09.896985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:09.896985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:09.896985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:09.896985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:09.896985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:09.896985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:09.896985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:09.896985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:09.896985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:09.896985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:09.896985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:09.896985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:09.897870 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:09.897870 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:09.897870 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:09.897870 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 301/591] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:09.898092 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e85f28-de65-4b9d-b9c4-ebbe98f12de8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:09.942819 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e8484-1306-45f9-bc2a-5825d9d650ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:09.943876 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e8484-1306-45f9-bc2a-5825d9d650ba None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:09.944356 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e8484-1306-45f9-bc2a-5825d9d650ba None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:09.945354 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e8484-1306-45f9-bc2a-5825d9d650ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:09.960994 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85e8484-1306-45f9-bc2a-5825d9d650ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:09.963487 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85e8484-1306-45f9-bc2a-5825d9d650ba None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:09.984121 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d85e8484-1306-45f9-bc2a-5825d9d650ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:09.984121 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:09.984121 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:09.984121 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:09.984121 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:09.984121 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:09.984121 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:09.984121 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:09.984121 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:09.984121 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:09.984121 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:09.984121 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:09.984121 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:09.984121 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:09.984121 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:09.984121 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:09.985119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:09.985119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:09.985119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:09.985119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:09.985119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:09.985119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:09.985119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:09.985119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:09.985119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:09.985119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:09.985119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:09.985119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:09.985119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:09.985119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:09.985119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:09.985119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:09.985119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:09.986018 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:09.986018 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:09.986018 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:09.986018 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 291/592] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:09.990646 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e8484-1306-45f9-bc2a-5825d9d650ba None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:10.030483 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e85f28-de65-4b9d-b9c4-ebbe98f12de8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:10.051152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:10.071882 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a757c8-14c3-4302-a9f5-9fc0ed7acdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:10.073047 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a757c8-14c3-4302-a9f5-9fc0ed7acdd3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:10.073685 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a757c8-14c3-4302-a9f5-9fc0ed7acdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:10.074311 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a757c8-14c3-4302-a9f5-9fc0ed7acdd3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:10.079706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:10.084102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a757c8-14c3-4302-a9f5-9fc0ed7acdd3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:10.085564 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a757c8-14c3-4302-a9f5-9fc0ed7acdd3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:10.095264 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e8484-1306-45f9-bc2a-5825d9d650ba None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:10.097465 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-05a757c8-14c3-4302-a9f5-9fc0ed7acdd3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 16:55:10.097465 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:10.097465 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:10.097465 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:10.097465 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:10.097465 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:10.097465 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:10.097465 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:10.097465 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:10.097465 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:10.097465 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:10.097465 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:10.097465 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:10.097465 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:10.097465 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:10.097465 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:10.098383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:10.098383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:10.098383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:10.098383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:10.098383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:10.098383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:10.098383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:10.098383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:10.098383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:10.098383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:10.098383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:10.098383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:10.098383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:10.098383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:10.098383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:10.098383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:10.098383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:10.098383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:10.099360 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:10.099360 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:10.099360 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:10.099360 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:10.099360 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:10.099360 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:10.099360 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:10.099360 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:10.099360 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 16:55:10.099360 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:10.099360 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 302/593] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:55:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:10.099999 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a757c8-14c3-4302-a9f5-9fc0ed7acdd3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:10.134413 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e041bd8-5967-4bc9-a3e5-a524816ffed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:10.135014 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e041bd8-5967-4bc9-a3e5-a524816ffed1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:10.135295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e041bd8-5967-4bc9-a3e5-a524816ffed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:10.135557 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e041bd8-5967-4bc9-a3e5-a524816ffed1 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:10.136224 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e041bd8-5967-4bc9-a3e5-a524816ffed1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:10.136586 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e041bd8-5967-4bc9-a3e5-a524816ffed1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:10.137950 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e041bd8-5967-4bc9-a3e5-a524816ffed1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:10.150153 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 292/594] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:10.156698 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f65c08-2512-45b5-92f7-20708cda877a None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:10.157053 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f65c08-2512-45b5-92f7-20708cda877a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:10.157403 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f65c08-2512-45b5-92f7-20708cda877a None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:10.159507 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 293/595] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:10.160058 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f65c08-2512-45b5-92f7-20708cda877a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:10.211665 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a757c8-14c3-4302-a9f5-9fc0ed7acdd3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:10.248979 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb23f0-30cd-4249-86ce-1b14e4878ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:10.249687 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb23f0-30cd-4249-86ce-1b14e4878ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:10.250041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb23f0-30cd-4249-86ce-1b14e4878ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:10.250403 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb23f0-30cd-4249-86ce-1b14e4878ad9 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:10.251116 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb23f0-30cd-4249-86ce-1b14e4878ad9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:10.251494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb23f0-30cd-4249-86ce-1b14e4878ad9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:10.252933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb23f0-30cd-4249-86ce-1b14e4878ad9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:10.265209 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 303/596] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:10.270395 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb23f0-30cd-4249-86ce-1b14e4878ad9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:10.280617 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f65c08-2512-45b5-92f7-20708cda877a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:10.309299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:10.319558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e55b7-7939-4c09-a11d-c71d637b2b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:10.320163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e55b7-7939-4c09-a11d-c71d637b2b8a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:10.320439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e55b7-7939-4c09-a11d-c71d637b2b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:10.320710 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e55b7-7939-4c09-a11d-c71d637b2b8a None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:10.321430 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57e55b7-7939-4c09-a11d-c71d637b2b8a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:10.321732 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57e55b7-7939-4c09-a11d-c71d637b2b8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:10.323200 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57e55b7-7939-4c09-a11d-c71d637b2b8a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:10.341135 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 294/597] 10.222.0.45 () {56 vars in 1137 bytes} [Wed Aug 10 16:55:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:10.342322 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f7005-b803-4c0c-9676-319a42c4d344 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:10.342632 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f7005-b803-4c0c-9676-319a42c4d344 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:10.342902 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f7005-b803-4c0c-9676-319a42c4d344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:10.350681 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:10.363259 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-579f7005-b803-4c0c-9676-319a42c4d344 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:10.364604 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-579f7005-b803-4c0c-9676-319a42c4d344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:10.384345 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb23f0-30cd-4249-86ce-1b14e4878ad9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:10.421859 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af3ea0-7ffe-44c6-ba88-1cea29bccb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:10.422515 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af3ea0-7ffe-44c6-ba88-1cea29bccb8e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:10.422797 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af3ea0-7ffe-44c6-ba88-1cea29bccb8e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:10.423071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af3ea0-7ffe-44c6-ba88-1cea29bccb8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:10.429879 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14af3ea0-7ffe-44c6-ba88-1cea29bccb8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:10.431386 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14af3ea0-7ffe-44c6-ba88-1cea29bccb8e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:10.448343 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 295/598] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:55:10.448343 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f7005-b803-4c0c-9676-319a42c4d344 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:10.449126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-14af3ea0-7ffe-44c6-ba88-1cea29bccb8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:10.449126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:10.449126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:10.449126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:10.449126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:10.449126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:10.449126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:10.449126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:10.449126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:10.449126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:10.449126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:10.449126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:10.449126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:10.449126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:10.449126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:10.449952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:10.449952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:10.449952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:10.449952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:10.449952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:10.449952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:10.449952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:10.449952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:10.449952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:10.449952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:10.449952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:10.449952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:10.449952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:10.449952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:10.449952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:10.449952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:10.449952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:10.450965 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:10.450965 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:10.450965 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:10.450965 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:10.450965 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:10.450965 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:10.450965 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:10.450965 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:10.450965 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 304/599] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:10.450965 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af3ea0-7ffe-44c6-ba88-1cea29bccb8e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:10.548274 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f7005-b803-4c0c-9676-319a42c4d344 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:10.554920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:10.562127 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af3ea0-7ffe-44c6-ba88-1cea29bccb8e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:10.595386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913ca90-c88f-4d66-8fd3-9626ec6f88b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:10.596195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913ca90-c88f-4d66-8fd3-9626ec6f88b3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:10.596571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913ca90-c88f-4d66-8fd3-9626ec6f88b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:10.596845 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913ca90-c88f-4d66-8fd3-9626ec6f88b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:10.597782 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2aff95-dbdf-457e-9ada-2c889a9ef6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:10.598653 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2aff95-dbdf-457e-9ada-2c889a9ef6c2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:10.599090 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2aff95-dbdf-457e-9ada-2c889a9ef6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:10.599458 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2aff95-dbdf-457e-9ada-2c889a9ef6c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:10.600119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:10.610818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f913ca90-c88f-4d66-8fd3-9626ec6f88b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:10.610935 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2aff95-dbdf-457e-9ada-2c889a9ef6c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:10.612047 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2aff95-dbdf-457e-9ada-2c889a9ef6c2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:10.612158 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f913ca90-c88f-4d66-8fd3-9626ec6f88b3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:10.626493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-f913ca90-c88f-4d66-8fd3-9626ec6f88b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:10.626493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:10.626493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:10.626493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:10.626493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:10.626493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:10.626493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:10.626493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:10.626493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:10.626493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:10.626493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:10.626493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:10.626493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:10.626493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:10.626493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:10.627610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:10.627610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:10.627610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:10.627610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:10.627610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:10.627610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:10.627610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:10.627610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:10.627610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:10.627610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:10.627610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:10.627610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:10.627610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:10.627610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:10.627610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:10.627610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:10.627610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:10.628704 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:10.628704 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:10.628704 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:10.628704 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:10.628704 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:10.628704 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:10.628704 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:10.628704 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:10.629492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-fd2aff95-dbdf-457e-9ada-2c889a9ef6c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:10.629492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:10.629492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:10.629492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:10.629492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:10.629492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:10.629492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:10.629492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:10.629492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:10.629492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:10.629492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:10.629492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:10.629492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:10.629492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:10.629492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:10.629492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:10.630567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:10.630567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:10.630567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:10.630567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:10.630567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:10.630567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:10.630567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:10.630567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:10.630567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:10.630567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:10.630567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:10.630567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:10.630567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:10.630567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:10.630567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:10.630567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:10.630567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:10.631660 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:10.631660 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:10.631660 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:10.632715 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 305/600] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:10.633528 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2aff95-dbdf-457e-9ada-2c889a9ef6c2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:10.639787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 296/601] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:10.640922 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913ca90-c88f-4d66-8fd3-9626ec6f88b3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:10.739576 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2aff95-dbdf-457e-9ada-2c889a9ef6c2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:10.767348 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913ca90-c88f-4d66-8fd3-9626ec6f88b3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:10.780196 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd76ad7-e4da-4c12-96dd-301149f2e07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:10.780922 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd76ad7-e4da-4c12-96dd-301149f2e07f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:10.781284 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd76ad7-e4da-4c12-96dd-301149f2e07f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:10.781632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd76ad7-e4da-4c12-96dd-301149f2e07f None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:10.782746 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd76ad7-e4da-4c12-96dd-301149f2e07f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:10.782862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd76ad7-e4da-4c12-96dd-301149f2e07f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:10.784350 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd76ad7-e4da-4c12-96dd-301149f2e07f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:10.802287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 306/602] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:10.802287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd76ad7-e4da-4c12-96dd-301149f2e07f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:10.809027 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743fec4d-5039-4ee2-8347-1b111b6b75b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:10.809590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:10.810029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fec4d-5039-4ee2-8347-1b111b6b75b4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:10.810706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fec4d-5039-4ee2-8347-1b111b6b75b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:10.811048 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fec4d-5039-4ee2-8347-1b111b6b75b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:10.821737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743fec4d-5039-4ee2-8347-1b111b6b75b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:10.827583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-743fec4d-5039-4ee2-8347-1b111b6b75b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:10.827583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:10.827583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:10.827583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:10.827583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:10.827583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:10.827583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:10.827583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:10.827583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:10.827583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:10.827583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:10.827583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:10.827583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:10.827583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:10.827583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:10.827583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:10.828522 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:10.828522 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:10.828522 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:10.828522 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:10.828522 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:10.828522 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:10.828522 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:10.828522 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:10.828522 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:10.828522 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:10.828522 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:10.828522 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:10.828522 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:10.828522 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:10.828522 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:10.828522 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:10.829357 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:10.829357 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:10.829357 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:10.829357 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:10.829357 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:10.829357 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:10.829357 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:10.829357 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:10.829357 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:10.829357 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:10.829357 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:10.829357 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:10.829357 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 297/603] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 10 16:55:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:10.830055 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743fec4d-5039-4ee2-8347-1b111b6b75b4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:10.891927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:10.906708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd76ad7-e4da-4c12-96dd-301149f2e07f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:10.938371 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743fec4d-5039-4ee2-8347-1b111b6b75b4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:10.943194 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb0915-907d-4e8e-8c04-beb2a117fdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:10.943814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb0915-907d-4e8e-8c04-beb2a117fdc9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:10.944087 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb0915-907d-4e8e-8c04-beb2a117fdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:10.944352 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb0915-907d-4e8e-8c04-beb2a117fdc9 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:10.944983 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb0915-907d-4e8e-8c04-beb2a117fdc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:10.945277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb0915-907d-4e8e-8c04-beb2a117fdc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:10.946676 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb0915-907d-4e8e-8c04-beb2a117fdc9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:10.958551 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 307/604] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:10.959317 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb0915-907d-4e8e-8c04-beb2a117fdc9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:10.978815 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db8f86-cbe0-45cc-9292-90a4c523edad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:10.979502 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db8f86-cbe0-45cc-9292-90a4c523edad None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:10.980072 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db8f86-cbe0-45cc-9292-90a4c523edad None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:10.980534 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db8f86-cbe0-45cc-9292-90a4c523edad None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:10.981345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db8f86-cbe0-45cc-9292-90a4c523edad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:10.981650 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db8f86-cbe0-45cc-9292-90a4c523edad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:10.983105 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db8f86-cbe0-45cc-9292-90a4c523edad None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:11.001589 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 298/605] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:11.001589 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db8f86-cbe0-45cc-9292-90a4c523edad None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:11.036070 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:11.071378 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb0915-907d-4e8e-8c04-beb2a117fdc9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:11.106273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db8f86-cbe0-45cc-9292-90a4c523edad None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:11.108549 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c282a2-9f90-4556-8565-7b1fad7af3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:11.109218 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c282a2-9f90-4556-8565-7b1fad7af3e5 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:11.109498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c282a2-9f90-4556-8565-7b1fad7af3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:11.109771 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c282a2-9f90-4556-8565-7b1fad7af3e5 None admin] PATH_INFO: `/v3/projects/d11f33af53374dffa61132642a7770f4/users/56d5c4afb9844fbbbe699f958deae6e4/roles/303e0fba379a41019ecc2e119d87fb1b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:11.121685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:11.127680 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c282a2-9f90-4556-8565-7b1fad7af3e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=d11f33af53374dffa61132642a7770f4, user_id=56d5c4afb9844fbbbe699f958deae6e4, role_id=303e0fba379a41019ecc2e119d87fb1b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:11.129200 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c282a2-9f90-4556-8565-7b1fad7af3e5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:11.142400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976edc6-7b2d-48b5-a0b7-0f5679fbca4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:11.142997 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976edc6-7b2d-48b5-a0b7-0f5679fbca4e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:11.143267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976edc6-7b2d-48b5-a0b7-0f5679fbca4e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:11.143529 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976edc6-7b2d-48b5-a0b7-0f5679fbca4e None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:11.144181 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976edc6-7b2d-48b5-a0b7-0f5679fbca4e None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:11.144475 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976edc6-7b2d-48b5-a0b7-0f5679fbca4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:11.145806 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976edc6-7b2d-48b5-a0b7-0f5679fbca4e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:11.155082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 299/606] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 10 16:55:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:11.155849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976edc6-7b2d-48b5-a0b7-0f5679fbca4e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:11.210626 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 308/607] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 10 16:55:10 2022] PUT /identity/v3/projects/d11f33af53374dffa61132642a7770f4/users/56d5c4afb9844fbbbe699f958deae6e4/roles/303e0fba379a41019ecc2e119d87fb1b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:11.211635 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c282a2-9f90-4556-8565-7b1fad7af3e5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:11.271054 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976edc6-7b2d-48b5-a0b7-0f5679fbca4e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:11.321213 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0efc800-6e2f-4d2a-bf28-3ec8906901a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:11.322604 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0efc800-6e2f-4d2a-bf28-3ec8906901a2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:11.324131 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0efc800-6e2f-4d2a-bf28-3ec8906901a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:11.324629 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0efc800-6e2f-4d2a-bf28-3ec8906901a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:11.334340 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0efc800-6e2f-4d2a-bf28-3ec8906901a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:11.336475 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0efc800-6e2f-4d2a-bf28-3ec8906901a2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:11.353719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-b0efc800-6e2f-4d2a-bf28-3ec8906901a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:11.353719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:11.353719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:11.353719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:11.353719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:11.353719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:11.353719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:11.353719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:11.353719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:11.353719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:11.353719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:11.353719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:11.353719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:11.353719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:11.353719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:11.355748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:11.355748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:11.355748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:11.355748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:11.355748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:11.355748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:11.355748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:11.355748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:11.355748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:11.355748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:11.355748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:11.355748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:11.355748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:11.355748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:11.355748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:11.355748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:11.355748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:11.356903 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:11.356903 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:11.356903 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:11.356903 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:11.356903 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:11.356903 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:11.356903 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:11.356903 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:11.364555 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 300/608] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:11.364555 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0efc800-6e2f-4d2a-bf28-3ec8906901a2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:11.366555 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c282a2-9f90-4556-8565-7b1fad7af3e5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:11.393417 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:11.411695 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:11.414501 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226735fa-0651-466a-8ceb-a4d287940cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:11.426379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226735fa-0651-466a-8ceb-a4d287940cf6 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:11.426379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226735fa-0651-466a-8ceb-a4d287940cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:11.426379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226735fa-0651-466a-8ceb-a4d287940cf6 None admin] PATH_INFO: `/v3/projects/d11f33af53374dffa61132642a7770f4/users/e1534cfef9e9428bb2e010fff200bc19/roles/303e0fba379a41019ecc2e119d87fb1b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:11.438771 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226735fa-0651-466a-8ceb-a4d287940cf6 None admin] RBAC: Authorizing `identity:create_grant(project_id=d11f33af53374dffa61132642a7770f4, user_id=e1534cfef9e9428bb2e010fff200bc19, role_id=303e0fba379a41019ecc2e119d87fb1b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:11.441081 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226735fa-0651-466a-8ceb-a4d287940cf6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:11.483405 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0efc800-6e2f-4d2a-bf28-3ec8906901a2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:11.519769 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f588f-e3ec-4fe4-a73e-1a5ce4b41baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:11.520476 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f588f-e3ec-4fe4-a73e-1a5ce4b41baa None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:11.520805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f588f-e3ec-4fe4-a73e-1a5ce4b41baa None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:11.521078 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f588f-e3ec-4fe4-a73e-1a5ce4b41baa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:11.532683 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f588f-e3ec-4fe4-a73e-1a5ce4b41baa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:11.534189 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f588f-e3ec-4fe4-a73e-1a5ce4b41baa None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:11.551210 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 309/609] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 10 16:55:11 2022] PUT /identity/v3/projects/d11f33af53374dffa61132642a7770f4/users/e1534cfef9e9428bb2e010fff200bc19/roles/303e0fba379a41019ecc2e119d87fb1b => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:11.552572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226735fa-0651-466a-8ceb-a4d287940cf6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:11.560530 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-b67f588f-e3ec-4fe4-a73e-1a5ce4b41baa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:11.560530 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:11.560530 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:11.560530 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:11.560530 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:11.560530 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:11.560530 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:11.560530 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:11.560530 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:11.560530 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:11.560530 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:11.560530 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:11.560530 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:11.560530 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:11.560530 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:11.560530 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:11.562159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:11.562159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:11.562159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:11.562159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:11.562159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:11.562159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:11.562159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:11.562159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:11.562159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:11.562159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:11.562159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:11.562159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:11.562159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:11.562159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:11.562159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:11.562159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:11.562159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:11.563212 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:11.563212 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:11.563212 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:11.563212 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 301/610] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:11.567435 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f588f-e3ec-4fe4-a73e-1a5ce4b41baa None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:11.717128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:11.721601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226735fa-0651-466a-8ceb-a4d287940cf6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:11.733803 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f588f-e3ec-4fe4-a73e-1a5ce4b41baa None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:11.762695 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e9d15-8971-4824-9fe4-aee2e38717f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:11.763302 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e9d15-8971-4824-9fe4-aee2e38717f2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:11.763683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e9d15-8971-4824-9fe4-aee2e38717f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:11.763961 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e9d15-8971-4824-9fe4-aee2e38717f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:11.764653 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e9d15-8971-4824-9fe4-aee2e38717f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:11.764960 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e9d15-8971-4824-9fe4-aee2e38717f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:11.767097 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e9d15-8971-4824-9fe4-aee2e38717f2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:11.780029 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 310/611] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:11.781263 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befd66c-71c3-41db-a96d-4adf596a7f36 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:11.781540 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befd66c-71c3-41db-a96d-4adf596a7f36 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:11.781805 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befd66c-71c3-41db-a96d-4adf596a7f36 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:11.783285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 311/612] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:11.785686 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bfa4f3-b8b7-4b29-afaf-b0042ebc19e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:11.786408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfa4f3-b8b7-4b29-afaf-b0042ebc19e4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:11.786692 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfa4f3-b8b7-4b29-afaf-b0042ebc19e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:11.787011 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfa4f3-b8b7-4b29-afaf-b0042ebc19e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:11.787173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1befd66c-71c3-41db-a96d-4adf596a7f36 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:11.787653 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bfa4f3-b8b7-4b29-afaf-b0042ebc19e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:11.787944 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bfa4f3-b8b7-4b29-afaf-b0042ebc19e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:11.789326 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bfa4f3-b8b7-4b29-afaf-b0042ebc19e4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:11.802756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 302/613] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:11.804058 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0526229-4c1e-419f-a324-d422d9f01580 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:11.804240 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0526229-4c1e-419f-a324-d422d9f01580 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:11.804509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0526229-4c1e-419f-a324-d422d9f01580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:11.824219 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:11.829840 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b0526229-4c1e-419f-a324-d422d9f01580 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:11.831179 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b0526229-4c1e-419f-a324-d422d9f01580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:11.907448 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 303/614] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:55:11.908144 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0526229-4c1e-419f-a324-d422d9f01580 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:11.923326 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1befd66c-71c3-41db-a96d-4adf596a7f36 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:11.969868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2143978-5b1d-4f84-ad3c-0c46574a9160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:11.971161 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2143978-5b1d-4f84-ad3c-0c46574a9160 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:11.971439 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2143978-5b1d-4f84-ad3c-0c46574a9160 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:11.972487 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2143978-5b1d-4f84-ad3c-0c46574a9160 None admin] PATH_INFO: `/v3/projects/d11f33af53374dffa61132642a7770f4/users/10f4f54890624da29ff217081445b6bf/roles/303e0fba379a41019ecc2e119d87fb1b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:11.994664 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2143978-5b1d-4f84-ad3c-0c46574a9160 None admin] RBAC: Authorizing `identity:create_grant(project_id=d11f33af53374dffa61132642a7770f4, user_id=10f4f54890624da29ff217081445b6bf, role_id=303e0fba379a41019ecc2e119d87fb1b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:11.997318 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2143978-5b1d-4f84-ad3c-0c46574a9160 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:12.008001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:12.029116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0526229-4c1e-419f-a324-d422d9f01580 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:12.064114 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341393a-47a4-4766-9397-e592239f7994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:12.064732 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341393a-47a4-4766-9397-e592239f7994 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:12.065006 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341393a-47a4-4766-9397-e592239f7994 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:12.065272 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341393a-47a4-4766-9397-e592239f7994 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:12.074288 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341393a-47a4-4766-9397-e592239f7994 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:12.075821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341393a-47a4-4766-9397-e592239f7994 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:12.082252 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 312/615] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 10 16:55:11 2022] PUT /identity/v3/projects/d11f33af53374dffa61132642a7770f4/users/10f4f54890624da29ff217081445b6bf/roles/303e0fba379a41019ecc2e119d87fb1b => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:12.084287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2143978-5b1d-4f84-ad3c-0c46574a9160 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:12.093800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-0341393a-47a4-4766-9397-e592239f7994 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:55:12.093800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:12.093800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:12.093800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:12.093800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:12.093800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:12.093800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:12.093800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:12.093800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:12.093800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:12.093800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:12.093800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:12.093800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:12.093800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:12.093800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:12.093800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:12.101465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:12.101465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:12.101465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:12.101465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:12.101465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:12.101465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:12.101465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:12.101465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:12.101465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:12.101465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:12.101465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:12.101465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:12.101465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:12.101465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:12.101465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:12.101465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:12.101465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:12.101465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:12.102620 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:12.102620 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:12.102620 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:12.102620 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:12.102620 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:12.102620 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:12.102620 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:12.102620 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:12.102620 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:55:12.102620 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:12.102620 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 304/616] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 10 16:55:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:12.102620 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341393a-47a4-4766-9397-e592239f7994 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:12.250550 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:12.264823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2143978-5b1d-4f84-ad3c-0c46574a9160 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:12.300130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341393a-47a4-4766-9397-e592239f7994 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:12.307914 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09ebe5-ff53-4b90-8d3a-90be9f47cc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:12.309182 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09ebe5-ff53-4b90-8d3a-90be9f47cc29 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:12.309468 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09ebe5-ff53-4b90-8d3a-90be9f47cc29 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:12.309790 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09ebe5-ff53-4b90-8d3a-90be9f47cc29 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:12.322484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f09ebe5-ff53-4b90-8d3a-90be9f47cc29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:12.328091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-8f09ebe5-ff53-4b90-8d3a-90be9f47cc29 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:12.328091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:12.328091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:12.328091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:12.328091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:12.328091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:12.328091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:12.328091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:12.328091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:12.328091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:12.328091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:12.328091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:12.328091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:12.328091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:12.328091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:12.328091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:12.329039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:12.329039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:12.329039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:12.329039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:12.329039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:12.329039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:12.329039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:12.329039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:12.329039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:12.329039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:12.329039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:12.329039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:12.329039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:12.329039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:12.329039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:12.329039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:12.329933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:12.329933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:12.329933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:12.329933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:12.329933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:12.329933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:12.329933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:12.329933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:12.329933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:12.329933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:12.329933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:12.329933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:12.333130 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 313/617] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 10 16:55:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:12.335615 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09ebe5-ff53-4b90-8d3a-90be9f47cc29 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:12.351469 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f680ced-9920-49f8-9eec-c5f4685c9b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:12.352538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f680ced-9920-49f8-9eec-c5f4685c9b81 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:12.353018 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f680ced-9920-49f8-9eec-c5f4685c9b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:12.353496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f680ced-9920-49f8-9eec-c5f4685c9b81 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:12.354748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f680ced-9920-49f8-9eec-c5f4685c9b81 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:12.355260 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f680ced-9920-49f8-9eec-c5f4685c9b81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:12.357045 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f680ced-9920-49f8-9eec-c5f4685c9b81 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:12.373679 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:12.390149 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 305/618] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 10 16:55:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:12.396736 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f680ced-9920-49f8-9eec-c5f4685c9b81 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:12.544838 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09ebe5-ff53-4b90-8d3a-90be9f47cc29 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:12.586370 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f680ced-9920-49f8-9eec-c5f4685c9b81 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:12.588052 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5a0e6-2bf9-40cc-b684-dc71b26d40a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:12.588876 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5a0e6-2bf9-40cc-b684-dc71b26d40a7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:12.589952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5a0e6-2bf9-40cc-b684-dc71b26d40a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:12.590471 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5a0e6-2bf9-40cc-b684-dc71b26d40a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:12.592107 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5a0e6-2bf9-40cc-b684-dc71b26d40a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:12.592637 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5a0e6-2bf9-40cc-b684-dc71b26d40a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:12.594210 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5a0e6-2bf9-40cc-b684-dc71b26d40a7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:12.600254 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:12.604353 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 314/619] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 10 16:55:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:12.612882 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5a0e6-2bf9-40cc-b684-dc71b26d40a7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:12.656295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd27e9e-fbac-4e73-9c2c-1eca164ae055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:12.657883 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd27e9e-fbac-4e73-9c2c-1eca164ae055 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:12.658167 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd27e9e-fbac-4e73-9c2c-1eca164ae055 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:12.658463 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd27e9e-fbac-4e73-9c2c-1eca164ae055 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:12.785519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:12.810232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd27e9e-fbac-4e73-9c2c-1eca164ae055 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:12.810232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd27e9e-fbac-4e73-9c2c-1eca164ae055 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:12.823073 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5a0e6-2bf9-40cc-b684-dc71b26d40a7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:12.877753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad62b9c3-ecd3-4e0b-b06a-775b4f000bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:12.878526 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad62b9c3-ecd3-4e0b-b06a-775b4f000bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:12.878909 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad62b9c3-ecd3-4e0b-b06a-775b4f000bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:12.879183 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad62b9c3-ecd3-4e0b-b06a-775b4f000bb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:12.901402 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad62b9c3-ecd3-4e0b-b06a-775b4f000bb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:12.903045 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad62b9c3-ecd3-4e0b-b06a-775b4f000bb0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:12.935915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ad62b9c3-ecd3-4e0b-b06a-775b4f000bb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:12.935915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:12.935915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:12.935915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:12.935915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:12.935915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:12.935915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:12.935915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:12.935915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:12.935915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:12.935915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:12.935915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:12.935915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:12.935915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:12.935915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:12.935915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:12.936957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:12.936957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:12.936957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:12.936957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:12.936957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:12.936957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:12.936957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:12.936957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:12.936957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:12.936957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:12.936957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:12.936957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:12.936957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:12.936957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:12.936957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:12.936957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:12.936957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:12.937851 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:12.937851 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:12.937851 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:12.937851 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 315/620] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:12.943318 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad62b9c3-ecd3-4e0b-b06a-775b4f000bb0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:12.979903 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 306/621] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:55:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:55:12.986383 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd27e9e-fbac-4e73-9c2c-1eca164ae055 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:13.069621 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:13.097580 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:13.104219 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd27e9e-fbac-4e73-9c2c-1eca164ae055 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:13.145855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a32bd-6b0d-48ae-aa31-3f04d6d7e025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:13.147058 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a32bd-6b0d-48ae-aa31-3f04d6d7e025 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:13.147444 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a32bd-6b0d-48ae-aa31-3f04d6d7e025 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:13.147823 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a32bd-6b0d-48ae-aa31-3f04d6d7e025 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:13.159908 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a32bd-6b0d-48ae-aa31-3f04d6d7e025 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:13.162057 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a32bd-6b0d-48ae-aa31-3f04d6d7e025 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:13.180468 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad62b9c3-ecd3-4e0b-b06a-775b4f000bb0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:13.181240 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-9f9a32bd-6b0d-48ae-aa31-3f04d6d7e025 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:13.181240 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:13.181240 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:13.181240 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:13.181240 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:13.181240 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:13.181240 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:13.181240 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:13.181240 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:13.181240 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:13.181240 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:13.181240 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:13.181240 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:13.181240 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:13.181240 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:13.181240 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:13.182345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:13.182345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:13.182345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:13.182345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:13.182345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:13.182345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:13.182345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:13.182345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:13.182345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:13.182345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:13.182345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:13.182345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:13.182345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:13.182345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:13.182345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:13.182345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:13.182345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:13.184034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:13.184034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:13.184034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:13.190373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 307/622] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:13.190373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a32bd-6b0d-48ae-aa31-3f04d6d7e025 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:13.280393 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b92ff-0575-4e21-bbdb-adf1552b03e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:13.281249 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b92ff-0575-4e21-bbdb-adf1552b03e4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:13.281677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b92ff-0575-4e21-bbdb-adf1552b03e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:13.282094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b92ff-0575-4e21-bbdb-adf1552b03e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:13.282904 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b92ff-0575-4e21-bbdb-adf1552b03e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:13.283384 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b92ff-0575-4e21-bbdb-adf1552b03e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:13.284899 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b92ff-0575-4e21-bbdb-adf1552b03e4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:13.307026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a32bd-6b0d-48ae-aa31-3f04d6d7e025 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:13.317785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 316/623] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:13.323036 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b92ff-0575-4e21-bbdb-adf1552b03e4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:13.346597 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17914c2-fb0c-4957-bc19-539edd7cf48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:13.347252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17914c2-fb0c-4957-bc19-539edd7cf48c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:13.347533 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17914c2-fb0c-4957-bc19-539edd7cf48c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:13.347799 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17914c2-fb0c-4957-bc19-539edd7cf48c None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:13.348505 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17914c2-fb0c-4957-bc19-539edd7cf48c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:13.348859 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17914c2-fb0c-4957-bc19-539edd7cf48c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:13.350245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17914c2-fb0c-4957-bc19-539edd7cf48c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:13.364322 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 308/624] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:13.371805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17914c2-fb0c-4957-bc19-539edd7cf48c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:13.383603 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:13.494787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17914c2-fb0c-4957-bc19-539edd7cf48c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:13.540882 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e3f98-3e70-4ddc-84dc-629cd1b37fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:13.541515 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e3f98-3e70-4ddc-84dc-629cd1b37fce None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:13.541796 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e3f98-3e70-4ddc-84dc-629cd1b37fce None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:13.542065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e3f98-3e70-4ddc-84dc-629cd1b37fce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:13.550138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b92ff-0575-4e21-bbdb-adf1552b03e4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:13.551453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e3f98-3e70-4ddc-84dc-629cd1b37fce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:13.554127 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e3f98-3e70-4ddc-84dc-629cd1b37fce None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:13.565135 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:13.569723 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-609e3f98-3e70-4ddc-84dc-629cd1b37fce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:13.569723 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:13.569723 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:13.569723 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:13.569723 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:13.569723 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:13.569723 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:13.569723 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:13.569723 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:13.569723 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:13.569723 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:13.569723 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:13.569723 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:13.569723 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:13.569723 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:13.570689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:13.570689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:13.570689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:13.570689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:13.570689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:13.570689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:13.570689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:13.570689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:13.570689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:13.570689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:13.570689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:13.570689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:13.570689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:13.570689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:13.570689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:13.570689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:13.570689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:13.571573 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:13.571573 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:13.571573 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:13.571573 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:13.571573 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:13.571573 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:13.571573 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:13.571573 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:13.571573 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 309/625] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:13.572569 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d123f-bf16-4697-a80a-985da753a8a1 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:13.572839 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d123f-bf16-4697-a80a-985da753a8a1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:13.573156 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d123f-bf16-4697-a80a-985da753a8a1 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:13.579086 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 310/626] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:13.579086 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d123f-bf16-4697-a80a-985da753a8a1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:13.603017 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6981b-a177-42f6-9052-03dd22ecfc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:13.604211 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6981b-a177-42f6-9052-03dd22ecfc8c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:13.604560 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6981b-a177-42f6-9052-03dd22ecfc8c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:13.604870 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6981b-a177-42f6-9052-03dd22ecfc8c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:13.615615 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6981b-a177-42f6-9052-03dd22ecfc8c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:13.618429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6981b-a177-42f6-9052-03dd22ecfc8c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:13.637683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5cd6981b-a177-42f6-9052-03dd22ecfc8c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 16:55:13.637683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:13.637683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:13.637683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:13.637683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:13.637683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:13.637683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:13.637683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:13.637683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:13.637683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:13.637683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:13.637683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:13.637683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:13.637683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:13.637683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:13.637683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:13.638740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:13.638740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:13.638740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:13.638740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:13.638740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:13.638740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:13.638740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:13.638740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:13.638740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:13.638740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:13.638740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:13.638740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:13.638740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:13.638740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:13.638740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:13.638740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:13.638740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:13.638740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:13.639770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:13.639770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:13.639770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:13.639770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:13.639770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:13.639770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:13.639770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:13.639770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:13.639770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 16:55:13.639770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:13.639770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 317/627] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:55:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:55:13.640886 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28379ec9-451b-496e-b474-44b9491b8f8a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:13.640886 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28379ec9-451b-496e-b474-44b9491b8f8a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:13.641055 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28379ec9-451b-496e-b474-44b9491b8f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:13.670547 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:13.671060 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-28379ec9-451b-496e-b474-44b9491b8f8a None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:13.672361 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-28379ec9-451b-496e-b474-44b9491b8f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:13.707054 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d123f-bf16-4697-a80a-985da753a8a1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:13.743662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2bac0-0a67-49eb-8a69-5e34fdc1f6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:13.744515 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2bac0-0a67-49eb-8a69-5e34fdc1f6c6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:13.744944 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2bac0-0a67-49eb-8a69-5e34fdc1f6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:13.745420 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2bac0-0a67-49eb-8a69-5e34fdc1f6c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:13.747002 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2bac0-0a67-49eb-8a69-5e34fdc1f6c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:13.747473 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2bac0-0a67-49eb-8a69-5e34fdc1f6c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:13.747692 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 318/628] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:55:13.748709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28379ec9-451b-496e-b474-44b9491b8f8a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:13.749589 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2bac0-0a67-49eb-8a69-5e34fdc1f6c6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:13.763208 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 311/629] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:13.775543 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2bac0-0a67-49eb-8a69-5e34fdc1f6c6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:13.857455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28379ec9-451b-496e-b474-44b9491b8f8a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:13.859136 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:13.888793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2bac0-0a67-49eb-8a69-5e34fdc1f6c6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:13.894340 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f583c3e-e382-4370-8d61-9e8771b5243a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:13.895395 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f583c3e-e382-4370-8d61-9e8771b5243a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:13.895854 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f583c3e-e382-4370-8d61-9e8771b5243a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:13.897093 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f583c3e-e382-4370-8d61-9e8771b5243a None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:13.898017 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f583c3e-e382-4370-8d61-9e8771b5243a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:13.898910 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f583c3e-e382-4370-8d61-9e8771b5243a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:13.902016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f583c3e-e382-4370-8d61-9e8771b5243a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:13.916199 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:13.925589 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b335c524-4d7c-47a7-93e5-97386af2c56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:13.928343 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335c524-4d7c-47a7-93e5-97386af2c56e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:13.928343 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335c524-4d7c-47a7-93e5-97386af2c56e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:13.928547 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 319/630] 10.222.0.45 () {56 vars in 1135 bytes} [Wed Aug 10 16:55:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:13.928827 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335c524-4d7c-47a7-93e5-97386af2c56e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:13.928926 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f583c3e-e382-4370-8d61-9e8771b5243a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:13.929000 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b335c524-4d7c-47a7-93e5-97386af2c56e None admin] RBAC: Adding query filter params (role.id=303e0fba379a41019ecc2e119d87fb1b, scope.project.id=d11f33af53374dffa61132642a7770f4, user.id=615cc36e21cc493b823e6cee695ac308) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:13.929000 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b335c524-4d7c-47a7-93e5-97386af2c56e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:13.929512 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b335c524-4d7c-47a7-93e5-97386af2c56e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:13.938040 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 312/631] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:55:13 2022] GET /identity/v3/role_assignments?user.id=615cc36e21cc493b823e6cee695ac308&scope.project.id=d11f33af53374dffa61132642a7770f4&role.id=303e0fba379a41019ecc2e119d87fb1b => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:13.939525 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfe2d6-ac1d-4db9-8d75-4318524403b8 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:13.940001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfe2d6-ac1d-4db9-8d75-4318524403b8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:13.940729 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfe2d6-ac1d-4db9-8d75-4318524403b8 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:13.942952 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 313/632] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:13.943935 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfe2d6-ac1d-4db9-8d75-4318524403b8 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:14.067429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f583c3e-e382-4370-8d61-9e8771b5243a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:14.097267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfe2d6-ac1d-4db9-8d75-4318524403b8 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:14.124994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4594fc-d62b-4cef-a286-55a143d8dad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:14.125563 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4594fc-d62b-4cef-a286-55a143d8dad8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:14.126129 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4594fc-d62b-4cef-a286-55a143d8dad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:14.127351 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4594fc-d62b-4cef-a286-55a143d8dad8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:14.148210 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4594fc-d62b-4cef-a286-55a143d8dad8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:14.148956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7180811-eb2e-4e36-8984-673bc009332c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:14.149842 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7180811-eb2e-4e36-8984-673bc009332c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:14.150111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7180811-eb2e-4e36-8984-673bc009332c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:14.150395 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7180811-eb2e-4e36-8984-673bc009332c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:14.154046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ab4594fc-d62b-4cef-a286-55a143d8dad8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:14.154046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:14.154046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:14.154046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:14.154046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:14.154046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:14.154046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:14.154046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:14.154046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:14.154046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:14.154046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:14.154046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:14.154046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:14.154046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:14.154046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:14.154046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:14.155156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:14.155156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:14.155156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:14.155156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:14.155156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:14.155156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:14.155156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:14.155156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:14.155156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:14.155156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:14.155156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:14.155156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:14.155156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:14.155156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:14.155156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:14.155156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:14.156011 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:14.156011 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:14.156011 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:14.156011 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:14.156011 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:14.156011 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:14.156011 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:14.156011 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:14.156011 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:14.156011 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:14.156011 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:14.156011 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:14.161522 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 320/633] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 10 16:55:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:14.163088 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc4293-ff26-4f8f-8616-2f1f0ae0ba3e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:14.163376 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc4293-ff26-4f8f-8616-2f1f0ae0ba3e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:14.163680 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc4293-ff26-4f8f-8616-2f1f0ae0ba3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:14.165157 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7180811-eb2e-4e36-8984-673bc009332c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:14.168885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7180811-eb2e-4e36-8984-673bc009332c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:14.185350 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-93bc4293-ff26-4f8f-8616-2f1f0ae0ba3e None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:14.185846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-e7180811-eb2e-4e36-8984-673bc009332c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:14.185846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:14.185846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:14.185846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:14.185846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:14.185846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:14.185846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:14.185846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:14.185846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:14.185846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:14.185846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:14.185846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:14.185846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:14.185846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:14.185846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:14.185846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:14.186781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:14.186781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:14.186781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:14.186781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:14.186781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:14.186781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:14.186781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:14.186781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:14.186781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:14.186781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:14.186781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:14.186781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:14.186781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:14.186781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:14.186781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:14.186781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:14.186781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:14.187690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:14.187690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:14.187690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:14.187866 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-93bc4293-ff26-4f8f-8616-2f1f0ae0ba3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:14.187939 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 314/634] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:14.188473 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7180811-eb2e-4e36-8984-673bc009332c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:14.210109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:14.247774 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:14.270520 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 321/635] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:55:14.271401 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc4293-ff26-4f8f-8616-2f1f0ae0ba3e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:14.329767 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7180811-eb2e-4e36-8984-673bc009332c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:14.376175 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3214b1-1fd2-4409-b526-688365a92288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:14.376710 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3214b1-1fd2-4409-b526-688365a92288 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:14.377298 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3214b1-1fd2-4409-b526-688365a92288 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:14.377568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3214b1-1fd2-4409-b526-688365a92288 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:14.378526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3214b1-1fd2-4409-b526-688365a92288 None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:14.378827 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3214b1-1fd2-4409-b526-688365a92288 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:14.380912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3214b1-1fd2-4409-b526-688365a92288 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:14.385784 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc4293-ff26-4f8f-8616-2f1f0ae0ba3e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:14.393215 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 315/636] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 10 16:55:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:14.394062 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3214b1-1fd2-4409-b526-688365a92288 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:14.427631 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f72d7-2227-4ffd-980c-7ae5ca8b7065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:14.428358 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f72d7-2227-4ffd-980c-7ae5ca8b7065 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:14.428707 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f72d7-2227-4ffd-980c-7ae5ca8b7065 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:14.429054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f72d7-2227-4ffd-980c-7ae5ca8b7065 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:14.429768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f72d7-2227-4ffd-980c-7ae5ca8b7065 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:14.430138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f72d7-2227-4ffd-980c-7ae5ca8b7065 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:14.431622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f72d7-2227-4ffd-980c-7ae5ca8b7065 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:14.444708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 322/637] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:14.445632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f72d7-2227-4ffd-980c-7ae5ca8b7065 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:14.458316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:14.514074 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3214b1-1fd2-4409-b526-688365a92288 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:14.522862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:14.555626 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f72d7-2227-4ffd-980c-7ae5ca8b7065 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:14.556676 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e8c3e-32cb-4793-b5fe-2bb2ef5c1c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:14.557306 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e8c3e-32cb-4793-b5fe-2bb2ef5c1c07 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:14.557585 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e8c3e-32cb-4793-b5fe-2bb2ef5c1c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:14.557860 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e8c3e-32cb-4793-b5fe-2bb2ef5c1c07 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:14.569105 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e8c3e-32cb-4793-b5fe-2bb2ef5c1c07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:14.574775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-767e8c3e-32cb-4793-b5fe-2bb2ef5c1c07 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:14.574775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:14.574775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:14.574775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:14.574775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:14.574775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:14.574775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:14.574775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:14.574775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:14.574775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:14.574775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:14.574775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:14.574775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:14.574775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:14.574775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:14.574775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:14.575756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:14.575756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:14.575756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:14.575756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:14.575756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:14.575756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:14.575756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:14.575756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:14.575756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:14.575756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:14.575756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:14.575756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:14.575756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:14.575756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:14.575756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:14.575756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:14.578375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:14.578375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:14.578375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:14.578375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:14.578375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:14.578375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:14.578375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:14.578375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:14.578375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:14.578375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:14.578375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:14.578375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:14.580605 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 316/638] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 10 16:55:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:14.581906 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ed9a7-bb13-4337-834b-47e22d673662 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:14.582194 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ed9a7-bb13-4337-834b-47e22d673662 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:14.582496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ed9a7-bb13-4337-834b-47e22d673662 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:14.586389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 317/639] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:14.586389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ed9a7-bb13-4337-834b-47e22d673662 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:14.602221 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef3193-853b-4c82-b4c3-3a2542f5bbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:14.602969 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef3193-853b-4c82-b4c3-3a2542f5bbf3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:14.603320 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef3193-853b-4c82-b4c3-3a2542f5bbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:14.603659 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef3193-853b-4c82-b4c3-3a2542f5bbf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:14.619386 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef3193-853b-4c82-b4c3-3a2542f5bbf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:14.621423 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef3193-853b-4c82-b4c3-3a2542f5bbf3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:14.638525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-deef3193-853b-4c82-b4c3-3a2542f5bbf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:14.638525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:14.638525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:14.638525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:14.638525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:14.638525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:14.638525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:14.638525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:14.638525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:14.638525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:14.638525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:14.638525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:14.638525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:14.638525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:14.638525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:14.638525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:14.639545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:14.639545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:14.639545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:14.639545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:14.639545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:14.639545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:14.639545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:14.639545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:14.639545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:14.639545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:14.639545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:14.639545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:14.639545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:14.639545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:14.639545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:14.639545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:14.639545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:14.640456 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:14.640456 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:14.640456 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:14.640456 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 323/640] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:14.640456 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef3193-853b-4c82-b4c3-3a2542f5bbf3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:14.706664 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:14.710029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ed9a7-bb13-4337-834b-47e22d673662 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:14.744969 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba4c3f-212e-44c0-85d5-48c50ccd42e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:14.745908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef3193-853b-4c82-b4c3-3a2542f5bbf3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:14.746215 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba4c3f-212e-44c0-85d5-48c50ccd42e5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:14.746437 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba4c3f-212e-44c0-85d5-48c50ccd42e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:14.746708 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba4c3f-212e-44c0-85d5-48c50ccd42e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:14.753904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba4c3f-212e-44c0-85d5-48c50ccd42e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:14.755366 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba4c3f-212e-44c0-85d5-48c50ccd42e5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:14.766949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-65ba4c3f-212e-44c0-85d5-48c50ccd42e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:14.766949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:14.766949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:14.766949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:14.766949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:14.766949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:14.766949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:14.766949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:14.766949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:14.766949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:14.766949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:14.766949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:14.766949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:14.766949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:14.766949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:14.767927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:14.767927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:14.767927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:14.767927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:14.767927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:14.767927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:14.767927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:14.767927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:14.767927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:14.767927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:14.767927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:14.767927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:14.767927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:14.767927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:14.767927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:14.767927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:14.767927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:14.768851 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:14.768851 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:14.768851 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:14.768851 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:14.768851 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:14.768851 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:14.768851 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:14.768851 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:14.768851 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 318/641] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:14.769358 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f0fa2-b569-4f1c-a4aa-857544f42726 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:14.769933 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f0fa2-b569-4f1c-a4aa-857544f42726 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:14.769933 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f0fa2-b569-4f1c-a4aa-857544f42726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:14.787163 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09d457-7df1-4b0e-96df-9853fc97be00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:14.787770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09d457-7df1-4b0e-96df-9853fc97be00 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:14.788046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09d457-7df1-4b0e-96df-9853fc97be00 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:14.788312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09d457-7df1-4b0e-96df-9853fc97be00 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:14.788424 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-da1f0fa2-b569-4f1c-a4aa-857544f42726 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:14.788969 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09d457-7df1-4b0e-96df-9853fc97be00 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:14.789267 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09d457-7df1-4b0e-96df-9853fc97be00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:14.789515 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-da1f0fa2-b569-4f1c-a4aa-857544f42726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:14.790984 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09d457-7df1-4b0e-96df-9853fc97be00 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:14.800297 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 324/642] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 10 16:55:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:14.801955 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09d457-7df1-4b0e-96df-9853fc97be00 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:14.810974 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:14.861687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 319/643] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:55:14.863251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f0fa2-b569-4f1c-a4aa-857544f42726 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:14.914311 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09d457-7df1-4b0e-96df-9853fc97be00 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:14.939532 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:14.951126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9032650-7287-4105-8053-be91b6e77cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:14.951729 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9032650-7287-4105-8053-be91b6e77cab None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:14.956743 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9032650-7287-4105-8053-be91b6e77cab None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:14.956743 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9032650-7287-4105-8053-be91b6e77cab None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:14.956743 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9032650-7287-4105-8053-be91b6e77cab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:14.956743 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9032650-7287-4105-8053-be91b6e77cab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:14.956743 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9032650-7287-4105-8053-be91b6e77cab None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:14.971599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 325/644] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:14.971754 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f0fa2-b569-4f1c-a4aa-857544f42726 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:14.971846 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9032650-7287-4105-8053-be91b6e77cab None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:15.006702 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41060cd7-64bb-447b-b14e-32dbf1be0123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:15.007302 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41060cd7-64bb-447b-b14e-32dbf1be0123 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:15.007573 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41060cd7-64bb-447b-b14e-32dbf1be0123 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:15.007855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41060cd7-64bb-447b-b14e-32dbf1be0123 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:15.008536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41060cd7-64bb-447b-b14e-32dbf1be0123 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:15.008845 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41060cd7-64bb-447b-b14e-32dbf1be0123 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:15.010272 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41060cd7-64bb-447b-b14e-32dbf1be0123 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:15.022477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 320/645] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:15.023278 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41060cd7-64bb-447b-b14e-32dbf1be0123 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:15.043504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:15.083209 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9032650-7287-4105-8053-be91b6e77cab None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:15.119807 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52409a-bb6e-4d50-8457-50faaf1aac41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:15.120471 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52409a-bb6e-4d50-8457-50faaf1aac41 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:15.120748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52409a-bb6e-4d50-8457-50faaf1aac41 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:15.121019 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52409a-bb6e-4d50-8457-50faaf1aac41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:15.125798 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41060cd7-64bb-447b-b14e-32dbf1be0123 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:15.130941 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a52409a-bb6e-4d50-8457-50faaf1aac41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:15.132386 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a52409a-bb6e-4d50-8457-50faaf1aac41 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:15.147314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4a52409a-bb6e-4d50-8457-50faaf1aac41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:15.147314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:15.147314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:15.147314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:15.147314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:15.147314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:15.147314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:15.147314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:15.147314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:15.147314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:15.147314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:15.147314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:15.147314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:15.147314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:15.147314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:15.147314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:15.148257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:15.148257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:15.148257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:15.148257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:15.148257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:15.148257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:15.148257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:15.148257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:15.148257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:15.148257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:15.148257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:15.148257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:15.148257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:15.148257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:15.148257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:15.148257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:15.148257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:15.149634 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:15.149634 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:15.149634 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:15.149634 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 326/646] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:15.149634 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52409a-bb6e-4d50-8457-50faaf1aac41 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:15.166766 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0715c-00c9-42db-b655-f97970ee4332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:15.167374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0715c-00c9-42db-b655-f97970ee4332 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:15.167762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0715c-00c9-42db-b655-f97970ee4332 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:15.167937 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0715c-00c9-42db-b655-f97970ee4332 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:15.178151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a0715c-00c9-42db-b655-f97970ee4332 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:15.183607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d9a0715c-00c9-42db-b655-f97970ee4332 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:15.183607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:15.183607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:15.183607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:15.183607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:15.183607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:15.183607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:15.183607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:15.183607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:15.183607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:15.183607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:15.183607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:15.183607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:15.183607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:15.183607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:15.183607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:15.184808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:15.184808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:15.184808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:15.184808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:15.184808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:15.184808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:15.184808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:15.184808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:15.184808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:15.184808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:15.184808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:15.184808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:15.184808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:15.184808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:15.184808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:15.184808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:15.185667 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:15.185667 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:15.185667 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:15.185667 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:15.185667 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:15.185667 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:15.185667 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:15.185667 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:15.185667 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:15.185667 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:15.185667 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:15.185667 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:15.185667 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 321/647] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 10 16:55:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:15.185667 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0715c-00c9-42db-b655-f97970ee4332 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:15.191304 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:15.267894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52409a-bb6e-4d50-8457-50faaf1aac41 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:15.269241 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:15.292289 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0715c-00c9-42db-b655-f97970ee4332 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:15.310938 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219fda4f-6635-41a5-86d3-7afb111075cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:15.311569 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fda4f-6635-41a5-86d3-7afb111075cb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:15.312271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fda4f-6635-41a5-86d3-7afb111075cb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:15.312550 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fda4f-6635-41a5-86d3-7afb111075cb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:15.321864 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219fda4f-6635-41a5-86d3-7afb111075cb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:15.323890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219fda4f-6635-41a5-86d3-7afb111075cb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:15.327149 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7836c-c8ae-436d-897c-2dc0019774f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:15.327896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7836c-c8ae-436d-897c-2dc0019774f0 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:15.328171 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7836c-c8ae-436d-897c-2dc0019774f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:15.328438 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7836c-c8ae-436d-897c-2dc0019774f0 None admin] PATH_INFO: `/v3/projects/d11f33af53374dffa61132642a7770f4/users/4986821ef9f84b99a835e387635aa9a8/roles/303e0fba379a41019ecc2e119d87fb1b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:15.334880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-219fda4f-6635-41a5-86d3-7afb111075cb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 16:55:15.334880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:15.334880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:15.334880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:15.334880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:15.334880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:15.334880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:15.334880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:15.334880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:15.334880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:15.334880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:15.334880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:15.334880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:15.334880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:15.334880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:15.334880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:15.335814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:15.335814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:15.335814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:15.335814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:15.335814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:15.335814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:15.335814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:15.335814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:15.335814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:15.335814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:15.335814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:15.335814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:15.335814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:15.335814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:15.335814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:15.335814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:15.335814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:15.335814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:15.337348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:15.337348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:15.337348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:15.337348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:15.337348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:15.337348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:15.337348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:15.337348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:15.337348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 16:55:15.337348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:15.337348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 327/648] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:55:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:15.337348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219fda4f-6635-41a5-86d3-7afb111075cb None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:15.349181 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7836c-c8ae-436d-897c-2dc0019774f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d11f33af53374dffa61132642a7770f4, user_id=4986821ef9f84b99a835e387635aa9a8, role_id=303e0fba379a41019ecc2e119d87fb1b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:15.350671 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7836c-c8ae-436d-897c-2dc0019774f0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:15.438846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 322/649] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 10 16:55:15 2022] PUT /identity/v3/projects/d11f33af53374dffa61132642a7770f4/users/4986821ef9f84b99a835e387635aa9a8/roles/303e0fba379a41019ecc2e119d87fb1b => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:15.438846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7836c-c8ae-436d-897c-2dc0019774f0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:15.447623 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:15.457640 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219fda4f-6635-41a5-86d3-7afb111075cb None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:15.505433 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c6ec6-2a44-430c-8658-9e36ce2429ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:15.506740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c6ec6-2a44-430c-8658-9e36ce2429ae None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:15.507052 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c6ec6-2a44-430c-8658-9e36ce2429ae None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:15.507327 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c6ec6-2a44-430c-8658-9e36ce2429ae None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:15.508012 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c6ec6-2a44-430c-8658-9e36ce2429ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:15.508314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c6ec6-2a44-430c-8658-9e36ce2429ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:15.509731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c6ec6-2a44-430c-8658-9e36ce2429ae None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:15.524058 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 328/650] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:15.525364 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c6ec6-2a44-430c-8658-9e36ce2429ae None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:15.554205 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7836c-c8ae-436d-897c-2dc0019774f0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:15.589709 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:15.602199 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2944d-2cc8-41f1-a34e-c3ce0b8e63e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:15.603182 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2944d-2cc8-41f1-a34e-c3ce0b8e63e5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:15.603469 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2944d-2cc8-41f1-a34e-c3ce0b8e63e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:15.603744 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2944d-2cc8-41f1-a34e-c3ce0b8e63e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:15.604439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d2944d-2cc8-41f1-a34e-c3ce0b8e63e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:15.605308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d2944d-2cc8-41f1-a34e-c3ce0b8e63e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:15.606835 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d2944d-2cc8-41f1-a34e-c3ce0b8e63e5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:15.616861 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 323/651] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 10 16:55:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:15.617544 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2944d-2cc8-41f1-a34e-c3ce0b8e63e5 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:15.683228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c6ec6-2a44-430c-8658-9e36ce2429ae None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:15.718698 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f7b4a-246b-4da8-843f-f32260437925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:15.719226 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f7b4a-246b-4da8-843f-f32260437925 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:15.719508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f7b4a-246b-4da8-843f-f32260437925 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:15.719783 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f7b4a-246b-4da8-843f-f32260437925 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:15.720512 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f7b4a-246b-4da8-843f-f32260437925 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:15.720820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f7b4a-246b-4da8-843f-f32260437925 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:15.722272 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f7b4a-246b-4da8-843f-f32260437925 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:15.740508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 329/652] 10.222.0.45 () {56 vars in 1137 bytes} [Wed Aug 10 16:55:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:15.746319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f7b4a-246b-4da8-843f-f32260437925 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:15.762229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2944d-2cc8-41f1-a34e-c3ce0b8e63e5 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:15.778758 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:15.807081 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e88f5c-f5c5-41a0-8ac5-73b826c8deb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:15.807729 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e88f5c-f5c5-41a0-8ac5-73b826c8deb5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:15.808787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e88f5c-f5c5-41a0-8ac5-73b826c8deb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:15.809063 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e88f5c-f5c5-41a0-8ac5-73b826c8deb5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:15.817612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e88f5c-f5c5-41a0-8ac5-73b826c8deb5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:15.819112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e88f5c-f5c5-41a0-8ac5-73b826c8deb5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:15.831287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-03e88f5c-f5c5-41a0-8ac5-73b826c8deb5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 16:55:15.831287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:15.831287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:15.831287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:15.831287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:15.831287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:15.831287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:15.831287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:15.831287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:15.831287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:15.831287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:15.831287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:15.831287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:15.831287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:15.831287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:15.831287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:15.832368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:15.832368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:15.832368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:15.832368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:15.832368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:15.832368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:15.832368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:15.832368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:15.832368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:15.832368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:15.832368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:15.832368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:15.832368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:15.832368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:15.832368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:15.832368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:15.832368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:15.832368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:15.834757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:15.834757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:15.834757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:15.834757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:15.834757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:15.834757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:15.834757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:15.834757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:15.834757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 16:55:15.834757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:15.834757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 324/653] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 10 16:55:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:15.834757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e88f5c-f5c5-41a0-8ac5-73b826c8deb5 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:15.868221 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f7b4a-246b-4da8-843f-f32260437925 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:15.891413 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:15.904318 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a0db82-0880-465d-9cd3-23110cb8eb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:15.907287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0db82-0880-465d-9cd3-23110cb8eb03 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:15.907287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0db82-0880-465d-9cd3-23110cb8eb03 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:15.907287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0db82-0880-465d-9cd3-23110cb8eb03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:15.916940 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a0db82-0880-465d-9cd3-23110cb8eb03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:15.919575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a0db82-0880-465d-9cd3-23110cb8eb03 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:15.936946 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-33a0db82-0880-465d-9cd3-23110cb8eb03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:15.936946 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:15.936946 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:15.936946 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:15.936946 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:15.936946 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:15.936946 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:15.936946 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:15.936946 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:15.936946 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:15.936946 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:15.936946 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:15.936946 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:15.936946 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:15.936946 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:15.936946 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:15.938143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:15.938143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:15.938143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:15.938143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:15.938143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:15.938143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:15.938143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:15.938143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:15.938143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:15.938143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:15.938143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:15.938143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:15.938143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:15.938143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:15.938143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:15.938143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:15.938143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:15.939157 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:15.939157 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:15.939157 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:15.939157 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 330/654] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:15.939402 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a0db82-0880-465d-9cd3-23110cb8eb03 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:15.960104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e88f5c-f5c5-41a0-8ac5-73b826c8deb5 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:16.000763 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f19147-8486-42a0-bf46-74e6b1c4132f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:16.001465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f19147-8486-42a0-bf46-74e6b1c4132f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:16.001838 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f19147-8486-42a0-bf46-74e6b1c4132f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:16.002208 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f19147-8486-42a0-bf46-74e6b1c4132f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:16.017269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:16.066913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a0db82-0880-465d-9cd3-23110cb8eb03 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:16.079671 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f19147-8486-42a0-bf46-74e6b1c4132f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:16.081185 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f19147-8486-42a0-bf46-74e6b1c4132f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:16.112443 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae67418f-b052-47e4-97a8-dadc2ae92a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:16.113777 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67418f-b052-47e4-97a8-dadc2ae92a86 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:16.114215 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67418f-b052-47e4-97a8-dadc2ae92a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:16.114901 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67418f-b052-47e4-97a8-dadc2ae92a86 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:16.115782 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae67418f-b052-47e4-97a8-dadc2ae92a86 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:16.116236 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae67418f-b052-47e4-97a8-dadc2ae92a86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:16.118955 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae67418f-b052-47e4-97a8-dadc2ae92a86 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:16.142748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:16.143489 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 331/655] 10.222.0.45 () {56 vars in 1139 bytes} [Wed Aug 10 16:55:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:16.144755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae67418f-b052-47e4-97a8-dadc2ae92a86 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:16.188246 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 325/656] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:55:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:55:16.189924 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f19147-8486-42a0-bf46-74e6b1c4132f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:16.266343 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae67418f-b052-47e4-97a8-dadc2ae92a86 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:16.290988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:16.297202 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f19147-8486-42a0-bf46-74e6b1c4132f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:16.309629 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34777245-00a3-4abd-91e8-8ddd49b9ad00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:16.310628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34777245-00a3-4abd-91e8-8ddd49b9ad00 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:16.310974 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34777245-00a3-4abd-91e8-8ddd49b9ad00 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:16.311311 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34777245-00a3-4abd-91e8-8ddd49b9ad00 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:16.312280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34777245-00a3-4abd-91e8-8ddd49b9ad00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:16.313095 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34777245-00a3-4abd-91e8-8ddd49b9ad00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:16.314831 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34777245-00a3-4abd-91e8-8ddd49b9ad00 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:16.331731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 332/657] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:16.332875 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34777245-00a3-4abd-91e8-8ddd49b9ad00 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:16.338215 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529c2aa-2eef-493b-b6d5-f74a1cc9dc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:16.339097 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529c2aa-2eef-493b-b6d5-f74a1cc9dc75 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:16.339574 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529c2aa-2eef-493b-b6d5-f74a1cc9dc75 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:16.339849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529c2aa-2eef-493b-b6d5-f74a1cc9dc75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:16.358843 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:16.413467 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c529c2aa-2eef-493b-b6d5-f74a1cc9dc75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:16.415240 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c529c2aa-2eef-493b-b6d5-f74a1cc9dc75 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:16.461435 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34777245-00a3-4abd-91e8-8ddd49b9ad00 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:16.498731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2334bc-4bf0-427a-996c-2df592f67758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:16.499373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2334bc-4bf0-427a-996c-2df592f67758 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:16.499647 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2334bc-4bf0-427a-996c-2df592f67758 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:16.499915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2334bc-4bf0-427a-996c-2df592f67758 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:16.511144 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2334bc-4bf0-427a-996c-2df592f67758 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:16.512599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2334bc-4bf0-427a-996c-2df592f67758 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:16.527559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-1f2334bc-4bf0-427a-996c-2df592f67758 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:16.527559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:16.527559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:16.527559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:16.527559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:16.527559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:16.527559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:16.527559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:16.527559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:16.527559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:16.527559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:16.527559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:16.527559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:16.527559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:16.527559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:16.527559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:16.528571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:16.528571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:16.528571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:16.528571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:16.528571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:16.528571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:16.528571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:16.528571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:16.528571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:16.528571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:16.528571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:16.528571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:16.528571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:16.528571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:16.528571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:16.528571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:16.528571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:16.529495 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:16.529495 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:16.529495 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:16.529495 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 333/658] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:16.530595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2966455-0e6f-4500-9632-9d304a46c850 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:16.530595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2966455-0e6f-4500-9632-9d304a46c850 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:16.530595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2966455-0e6f-4500-9632-9d304a46c850 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:16.534533 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 334/659] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:16.534533 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2966455-0e6f-4500-9632-9d304a46c850 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:16.544505 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 326/660] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:55:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:55:16.545384 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529c2aa-2eef-493b-b6d5-f74a1cc9dc75 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:16.579659 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:16.635217 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:16.699719 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2966455-0e6f-4500-9632-9d304a46c850 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:16.725822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529c2aa-2eef-493b-b6d5-f74a1cc9dc75 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:16.750656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8ebfc-cc9b-42cb-8086-108539481ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:16.751293 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8ebfc-cc9b-42cb-8086-108539481ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:16.751577 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8ebfc-cc9b-42cb-8086-108539481ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:16.752351 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8ebfc-cc9b-42cb-8086-108539481ac6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:16.765552 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed8ebfc-cc9b-42cb-8086-108539481ac6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:16.767098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed8ebfc-cc9b-42cb-8086-108539481ac6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:16.775016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f36491-72ee-4c30-aa87-56527ae947fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:16.775773 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f36491-72ee-4c30-aa87-56527ae947fa None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:16.776059 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f36491-72ee-4c30-aa87-56527ae947fa None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:16.776359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f36491-72ee-4c30-aa87-56527ae947fa None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:16.778255 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f36491-72ee-4c30-aa87-56527ae947fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:16.778589 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f36491-72ee-4c30-aa87-56527ae947fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:16.780545 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f36491-72ee-4c30-aa87-56527ae947fa None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:16.783731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2ed8ebfc-cc9b-42cb-8086-108539481ac6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 16:55:16.783731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:16.783731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:16.783731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:16.783731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:16.783731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:16.783731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:16.783731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:16.783731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:16.783731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:16.783731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:16.783731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:16.783731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:16.783731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:16.783731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:16.783731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:16.784994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:16.784994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:16.784994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:16.784994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:16.784994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:16.784994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:16.784994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:16.784994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:16.784994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:16.784994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:16.784994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:16.784994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:16.784994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:16.784994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:16.784994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:16.784994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:16.784994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:16.784994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:16.786319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:16.786319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:16.786319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:16.786319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:16.786319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:16.786319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:16.786319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:16.786319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:16.786319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 16:55:16.786319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:16.793011 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 335/661] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:55:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:16.794493 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79227cf2-3e09-489b-aacc-a153546e9c33 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:16.794889 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79227cf2-3e09-489b-aacc-a153546e9c33 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:16.795260 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79227cf2-3e09-489b-aacc-a153546e9c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:16.800311 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 327/662] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:16.800311 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f36491-72ee-4c30-aa87-56527ae947fa None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:16.815747 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-79227cf2-3e09-489b-aacc-a153546e9c33 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:16.817253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-79227cf2-3e09-489b-aacc-a153546e9c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:16.919365 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 336/663] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:55:16.920430 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79227cf2-3e09-489b-aacc-a153546e9c33 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:16.924878 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f36491-72ee-4c30-aa87-56527ae947fa None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:16.936568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:16.960223 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:16.964144 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd6df4-c44b-498d-8706-15a6471cb250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:16.964922 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd6df4-c44b-498d-8706-15a6471cb250 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:16.965238 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd6df4-c44b-498d-8706-15a6471cb250 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:16.965509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd6df4-c44b-498d-8706-15a6471cb250 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:16.981829 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd6df4-c44b-498d-8706-15a6471cb250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:16.983321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd6df4-c44b-498d-8706-15a6471cb250 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:16.999233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-65dd6df4-c44b-498d-8706-15a6471cb250 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:16.999233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:16.999233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:16.999233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:16.999233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:16.999233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:16.999233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:16.999233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:16.999233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:16.999233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:16.999233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:16.999233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:16.999233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:16.999233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:16.999233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:16.999233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:17.000309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:17.000309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:17.000309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:17.000309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:17.000309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:17.000309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:17.000309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:17.000309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:17.000309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:17.000309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:17.000309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:17.000309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:17.000309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:17.000309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:17.000309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:17.000309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:17.000309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:17.001207 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:17.001207 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:17.001207 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:17.001207 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 328/664] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:17.001452 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd6df4-c44b-498d-8706-15a6471cb250 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:17.051158 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79227cf2-3e09-489b-aacc-a153546e9c33 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:17.088614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee8466-9150-4e6a-bb19-ee2c9f6dea19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:17.089759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee8466-9150-4e6a-bb19-ee2c9f6dea19 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:17.090033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee8466-9150-4e6a-bb19-ee2c9f6dea19 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:17.090324 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee8466-9150-4e6a-bb19-ee2c9f6dea19 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:17.091191 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee8466-9150-4e6a-bb19-ee2c9f6dea19 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:17.091454 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee8466-9150-4e6a-bb19-ee2c9f6dea19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:17.092898 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee8466-9150-4e6a-bb19-ee2c9f6dea19 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:17.112600 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd6df4-c44b-498d-8706-15a6471cb250 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:17.122311 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 337/665] 10.222.0.45 () {56 vars in 1137 bytes} [Wed Aug 10 16:55:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:17.122311 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee8466-9150-4e6a-bb19-ee2c9f6dea19 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:17.148702 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af7d39-5e20-4156-95ee-a969da6b38af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:17.149594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af7d39-5e20-4156-95ee-a969da6b38af None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:17.149868 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af7d39-5e20-4156-95ee-a969da6b38af None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:17.150141 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af7d39-5e20-4156-95ee-a969da6b38af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:17.157942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af7d39-5e20-4156-95ee-a969da6b38af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:17.159455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af7d39-5e20-4156-95ee-a969da6b38af None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:17.172577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-10af7d39-5e20-4156-95ee-a969da6b38af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:17.172577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:17.172577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:17.172577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:17.172577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:17.172577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:17.172577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:17.172577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:17.172577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:17.172577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:17.172577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:17.172577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:17.172577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:17.172577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:17.172577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:17.173532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:17.173532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:17.173532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:17.173532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:17.173532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:17.173532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:17.173532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:17.173532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:17.173532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:17.173532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:17.173532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:17.173532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:17.173532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:17.173532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:17.173532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:17.173532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:17.173532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:17.175373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:17.175373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:17.175373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:17.175373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:17.175373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:17.175373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:17.175373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:17.175373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:17.175373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 329/666] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:17.175373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af7d39-5e20-4156-95ee-a969da6b38af None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:17.208478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:17.221582 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:17.238616 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee8466-9150-4e6a-bb19-ee2c9f6dea19 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:17.276903 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c909fc-63fa-462e-98ed-7fa42f312173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:17.277547 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c909fc-63fa-462e-98ed-7fa42f312173 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:17.277826 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c909fc-63fa-462e-98ed-7fa42f312173 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:17.278096 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c909fc-63fa-462e-98ed-7fa42f312173 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:17.291051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c909fc-63fa-462e-98ed-7fa42f312173 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:17.296902 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-71c909fc-63fa-462e-98ed-7fa42f312173 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:55:17.296902 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:17.296902 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:17.296902 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:17.296902 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:17.296902 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:17.296902 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:17.296902 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:17.296902 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:17.296902 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:17.296902 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:17.296902 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:17.296902 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:17.296902 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:17.296902 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:17.296902 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:17.297891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:17.297891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:17.297891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:17.297891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:17.297891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:17.297891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:17.297891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:17.297891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:17.297891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:17.297891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:17.297891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:17.297891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:17.297891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:17.297891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:17.297891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:17.297891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:17.298759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:17.298759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:17.298759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:17.298759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:17.298759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:17.298759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:17.298759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:17.298759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:17.298759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:17.298759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:17.298759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:55:17.298759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:17.298759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 338/667] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:55:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:17.300232 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c909fc-63fa-462e-98ed-7fa42f312173 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:17.311955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af7d39-5e20-4156-95ee-a969da6b38af None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:17.348057 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc34ef7b-a95f-457d-9e82-1b4f2c500fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:17.348711 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34ef7b-a95f-457d-9e82-1b4f2c500fcd None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:17.348991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34ef7b-a95f-457d-9e82-1b4f2c500fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:17.349256 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34ef7b-a95f-457d-9e82-1b4f2c500fcd None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:17.349900 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc34ef7b-a95f-457d-9e82-1b4f2c500fcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:17.350197 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc34ef7b-a95f-457d-9e82-1b4f2c500fcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:17.351621 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc34ef7b-a95f-457d-9e82-1b4f2c500fcd None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:17.368797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 330/668] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:17.368797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc34ef7b-a95f-457d-9e82-1b4f2c500fcd None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:17.414324 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c909fc-63fa-462e-98ed-7fa42f312173 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:17.450919 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d28825-211d-4cc6-8823-cb0c4d23d1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:17.451524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d28825-211d-4cc6-8823-cb0c4d23d1fa None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:17.451800 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d28825-211d-4cc6-8823-cb0c4d23d1fa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:17.452111 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d28825-211d-4cc6-8823-cb0c4d23d1fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:17.456761 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:17.467228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:17.477751 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc34ef7b-a95f-457d-9e82-1b4f2c500fcd None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:17.514743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ef2ca-f08a-4df9-bef8-3de224d90031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:17.515347 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ef2ca-f08a-4df9-bef8-3de224d90031 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:17.515622 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ef2ca-f08a-4df9-bef8-3de224d90031 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:17.515892 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ef2ca-f08a-4df9-bef8-3de224d90031 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:17.516576 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ef2ca-f08a-4df9-bef8-3de224d90031 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:17.516878 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ef2ca-f08a-4df9-bef8-3de224d90031 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:17.518312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ef2ca-f08a-4df9-bef8-3de224d90031 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:17.522563 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d28825-211d-4cc6-8823-cb0c4d23d1fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:17.524302 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d28825-211d-4cc6-8823-cb0c4d23d1fa None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:17.536697 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 331/669] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:17.536697 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ef2ca-f08a-4df9-bef8-3de224d90031 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:17.635209 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 339/670] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:55:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:55:17.636207 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d28825-211d-4cc6-8823-cb0c4d23d1fa None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:17.654055 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ef2ca-f08a-4df9-bef8-3de224d90031 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:17.685908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:17.697863 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96f2db-8ed9-486a-be1d-1d297fdbda1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:17.701901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96f2db-8ed9-486a-be1d-1d297fdbda1b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:17.701901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96f2db-8ed9-486a-be1d-1d297fdbda1b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:17.701901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96f2db-8ed9-486a-be1d-1d297fdbda1b None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:17.701901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd96f2db-8ed9-486a-be1d-1d297fdbda1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:17.701901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd96f2db-8ed9-486a-be1d-1d297fdbda1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:17.701901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd96f2db-8ed9-486a-be1d-1d297fdbda1b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:17.709934 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 332/671] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:55:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:17.710900 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96f2db-8ed9-486a-be1d-1d297fdbda1b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:17.713578 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:17.742120 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d28825-211d-4cc6-8823-cb0c4d23d1fa None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:17.778255 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331aac21-273a-4acb-a522-2bd0408595d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:17.778908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331aac21-273a-4acb-a522-2bd0408595d2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:17.779194 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331aac21-273a-4acb-a522-2bd0408595d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:17.779453 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331aac21-273a-4acb-a522-2bd0408595d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:17.786119 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331aac21-273a-4acb-a522-2bd0408595d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:17.787597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331aac21-273a-4acb-a522-2bd0408595d2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:17.798738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-331aac21-273a-4acb-a522-2bd0408595d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:17.798738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:17.798738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:17.798738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:17.798738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:17.798738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:17.798738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:17.798738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:17.798738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:17.798738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:17.798738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:17.798738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:17.798738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:17.798738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:17.798738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:17.799840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:17.799840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:17.799840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:17.799840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:17.799840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:17.799840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:17.799840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:17.799840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:17.799840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:17.799840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:17.799840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:17.799840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:17.799840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:17.799840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:17.799840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:17.799840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:17.799840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:17.800846 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:17.800846 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:17.800846 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:17.800846 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:17.800846 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:17.800846 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:17.800846 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:17.800846 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:17.800846 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 340/672] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:17.801338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478c229-43d4-4d3b-83d1-480239fa348a None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:17.801685 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478c229-43d4-4d3b-83d1-480239fa348a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:17.801871 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478c229-43d4-4d3b-83d1-480239fa348a None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:17.803126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 341/673] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:17.803817 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478c229-43d4-4d3b-83d1-480239fa348a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:17.815701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96f2db-8ed9-486a-be1d-1d297fdbda1b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:17.849542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e87fd0-754d-4b21-b8af-bb33334e9201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:17.850143 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e87fd0-754d-4b21-b8af-bb33334e9201 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:17.850448 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e87fd0-754d-4b21-b8af-bb33334e9201 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:17.850716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e87fd0-754d-4b21-b8af-bb33334e9201 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:17.851436 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e87fd0-754d-4b21-b8af-bb33334e9201 None admin] RBAC: Adding query filter params (role.id=303e0fba379a41019ecc2e119d87fb1b, scope.project.id=d11f33af53374dffa61132642a7770f4, user.id=56d5c4afb9844fbbbe699f958deae6e4) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:17.851740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e87fd0-754d-4b21-b8af-bb33334e9201 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:17.853209 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e87fd0-754d-4b21-b8af-bb33334e9201 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:17.859098 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 333/674] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:55:17 2022] GET /identity/v3/role_assignments?user.id=56d5c4afb9844fbbbe699f958deae6e4&scope.project.id=d11f33af53374dffa61132642a7770f4&role.id=303e0fba379a41019ecc2e119d87fb1b => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:17.859955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e87fd0-754d-4b21-b8af-bb33334e9201 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:17.915254 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478c229-43d4-4d3b-83d1-480239fa348a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:17.918289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:17.941500 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:17.958367 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7b7c9-5ec6-46f5-8ba4-1d56d2322275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:17.959035 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7b7c9-5ec6-46f5-8ba4-1d56d2322275 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:17.959313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7b7c9-5ec6-46f5-8ba4-1d56d2322275 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:17.959587 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7b7c9-5ec6-46f5-8ba4-1d56d2322275 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:17.971274 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7b7c9-5ec6-46f5-8ba4-1d56d2322275 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:17.972751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7b7c9-5ec6-46f5-8ba4-1d56d2322275 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:17.984743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e87fd0-754d-4b21-b8af-bb33334e9201 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:17.990339 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-73f7b7c9-5ec6-46f5-8ba4-1d56d2322275 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:17.990339 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:17.990339 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:17.990339 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:17.990339 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:17.990339 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:17.990339 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:17.990339 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:17.990339 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:17.990339 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:17.990339 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:17.990339 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:17.990339 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:17.990339 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:17.990339 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:17.990339 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:17.991375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:17.991375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:17.991375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:17.991375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:17.991375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:17.991375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:17.991375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:17.991375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:17.991375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:17.991375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:17.991375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:17.991375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:17.991375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:17.991375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:17.991375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:17.991375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:17.991375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:17.992276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:17.992276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:17.992276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:17.994734 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 342/675] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:17.997350 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7b7c9-5ec6-46f5-8ba4-1d56d2322275 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:18.025948 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b1fb22-da6c-4bc2-9e02-b7d399e9bb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:18.027418 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1fb22-da6c-4bc2-9e02-b7d399e9bb8e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:18.027418 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1fb22-da6c-4bc2-9e02-b7d399e9bb8e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:18.027418 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1fb22-da6c-4bc2-9e02-b7d399e9bb8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:18.039608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b1fb22-da6c-4bc2-9e02-b7d399e9bb8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:18.041078 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b1fb22-da6c-4bc2-9e02-b7d399e9bb8e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:18.056531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-c1b1fb22-da6c-4bc2-9e02-b7d399e9bb8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:18.056531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:18.056531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:18.056531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:18.056531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:18.056531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:18.056531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:18.056531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:18.056531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:18.056531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:18.056531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:18.056531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:18.056531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:18.056531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:18.056531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:18.056531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:18.063371 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:18.063371 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:18.063371 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:18.063371 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:18.063371 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:18.063371 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:18.063371 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:18.063371 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:18.063371 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:18.063371 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:18.063371 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:18.063371 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:18.063371 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:18.063371 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:18.063371 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:18.063371 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:18.063371 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:18.064294 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:18.064294 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:18.064294 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:18.064294 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 334/676] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:18.064294 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77616f78-2f56-439a-8550-052f16579fd9 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:18.064294 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77616f78-2f56-439a-8550-052f16579fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:18.064294 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77616f78-2f56-439a-8550-052f16579fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:18.082854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-77616f78-2f56-439a-8550-052f16579fd9 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:18.084065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-77616f78-2f56-439a-8550-052f16579fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:18.117996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7b7c9-5ec6-46f5-8ba4-1d56d2322275 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:18.157585 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4a741-07b7-44b3-9c16-12585fa5eef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:18.158243 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4a741-07b7-44b3-9c16-12585fa5eef1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:18.158588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4a741-07b7-44b3-9c16-12585fa5eef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:18.158878 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4a741-07b7-44b3-9c16-12585fa5eef1 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:18.159576 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4a741-07b7-44b3-9c16-12585fa5eef1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:18.159882 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4a741-07b7-44b3-9c16-12585fa5eef1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:18.160533 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 335/677] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:55:18.161324 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4a741-07b7-44b3-9c16-12585fa5eef1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:18.163551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77616f78-2f56-439a-8550-052f16579fd9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:18.178403 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 343/678] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:18.178403 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4a741-07b7-44b3-9c16-12585fa5eef1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:18.200214 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:18.227180 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:18.284965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77616f78-2f56-439a-8550-052f16579fd9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:18.292390 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4a741-07b7-44b3-9c16-12585fa5eef1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:18.334408 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929f03b-9066-442e-ac76-c927865c6b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:18.335436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929f03b-9066-442e-ac76-c927865c6b42 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:18.335931 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929f03b-9066-442e-ac76-c927865c6b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:18.336393 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929f03b-9066-442e-ac76-c927865c6b42 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:18.337290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929f03b-9066-442e-ac76-c927865c6b42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:18.337815 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929f03b-9066-442e-ac76-c927865c6b42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:18.340138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929f03b-9066-442e-ac76-c927865c6b42 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:18.353537 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 344/679] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:18.355052 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929f03b-9066-442e-ac76-c927865c6b42 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:18.440391 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83ab1f-3a25-4f29-b166-f053fd6ca43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:18.440912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83ab1f-3a25-4f29-b166-f053fd6ca43a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:18.441198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83ab1f-3a25-4f29-b166-f053fd6ca43a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:18.441457 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83ab1f-3a25-4f29-b166-f053fd6ca43a None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:18.442106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83ab1f-3a25-4f29-b166-f053fd6ca43a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:18.442430 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83ab1f-3a25-4f29-b166-f053fd6ca43a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:18.443815 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83ab1f-3a25-4f29-b166-f053fd6ca43a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:18.452444 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:18.457040 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 336/680] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:18.458172 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83ab1f-3a25-4f29-b166-f053fd6ca43a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:18.481788 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929f03b-9066-442e-ac76-c927865c6b42 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:18.519544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bca52b-9a17-4c6b-bfd3-e9551cbecf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:18.520256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bca52b-9a17-4c6b-bfd3-e9551cbecf16 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:18.520534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bca52b-9a17-4c6b-bfd3-e9551cbecf16 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:18.520813 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bca52b-9a17-4c6b-bfd3-e9551cbecf16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:18.521581 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bca52b-9a17-4c6b-bfd3-e9551cbecf16 None admin] RBAC: Adding query filter params (role.id=303e0fba379a41019ecc2e119d87fb1b, scope.project.id=d11f33af53374dffa61132642a7770f4, user.id=e1534cfef9e9428bb2e010fff200bc19) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:18.521892 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bca52b-9a17-4c6b-bfd3-e9551cbecf16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:18.523502 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bca52b-9a17-4c6b-bfd3-e9551cbecf16 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:18.538399 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 345/681] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:55:18 2022] GET /identity/v3/role_assignments?user.id=e1534cfef9e9428bb2e010fff200bc19&scope.project.id=d11f33af53374dffa61132642a7770f4&role.id=303e0fba379a41019ecc2e119d87fb1b => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:18.538399 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bca52b-9a17-4c6b-bfd3-e9551cbecf16 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:18.576182 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83ab1f-3a25-4f29-b166-f053fd6ca43a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:18.587432 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:18.628029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0038a29-d809-46f4-8500-68d8570d9c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:18.629107 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0038a29-d809-46f4-8500-68d8570d9c81 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:18.629451 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0038a29-d809-46f4-8500-68d8570d9c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:18.629787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0038a29-d809-46f4-8500-68d8570d9c81 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:18.648218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0038a29-d809-46f4-8500-68d8570d9c81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:18.654765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-f0038a29-d809-46f4-8500-68d8570d9c81 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:18.654765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:18.654765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:18.654765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:18.654765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:18.654765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:18.654765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:18.654765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:18.654765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:18.654765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:18.654765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:18.654765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:18.654765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:18.654765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:18.654765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:18.654765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:18.655831 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:18.655831 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:18.655831 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:18.655831 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:18.655831 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:18.655831 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:18.655831 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:18.655831 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:18.655831 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:18.655831 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:18.655831 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:18.655831 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:18.655831 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:18.655831 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:18.655831 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:18.655831 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:18.656685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:18.656685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:18.656685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:18.656685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:18.656685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:18.656685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:18.656685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:18.656685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:18.656685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:18.656685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:18.656685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:18.656685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:18.661171 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 337/682] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 10 16:55:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:18.662221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0038a29-d809-46f4-8500-68d8570d9c81 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:18.682121 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bca52b-9a17-4c6b-bfd3-e9551cbecf16 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:18.730585 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7b4e1-ef3e-4be4-8517-2403454c7032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:18.731152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7b4e1-ef3e-4be4-8517-2403454c7032 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:18.731425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7b4e1-ef3e-4be4-8517-2403454c7032 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:18.731688 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7b4e1-ef3e-4be4-8517-2403454c7032 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:18.740267 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7b4e1-ef3e-4be4-8517-2403454c7032 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:18.741938 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7b4e1-ef3e-4be4-8517-2403454c7032 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:18.745953 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:18.754924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4ae7b4e1-ef3e-4be4-8517-2403454c7032 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:55:18.754924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:18.754924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:18.754924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:18.754924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:18.754924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:18.754924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:18.754924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:18.754924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:18.754924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:18.754924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:18.754924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:18.754924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:18.754924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:18.754924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:18.754924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:18.762490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:18.762490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:18.762490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:18.762490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:18.762490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:18.762490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:18.762490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:18.762490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:18.762490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:18.762490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:18.762490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:18.762490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:18.762490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:18.762490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:18.762490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:18.762490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:18.762490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:18.762490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:18.763445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:18.763445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:18.763445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:18.763445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:18.763445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:18.763445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:18.763445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:18.763445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:18.763445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:55:18.763445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:18.763445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 346/683] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 10 16:55:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:18.763445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7b4e1-ef3e-4be4-8517-2403454c7032 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:18.784248 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0038a29-d809-46f4-8500-68d8570d9c81 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:18.830797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc54678c-f634-4ab6-b707-47f0f806be7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:18.831694 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54678c-f634-4ab6-b707-47f0f806be7d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:18.831965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54678c-f634-4ab6-b707-47f0f806be7d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:18.832470 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54678c-f634-4ab6-b707-47f0f806be7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:18.842164 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc54678c-f634-4ab6-b707-47f0f806be7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:18.844701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc54678c-f634-4ab6-b707-47f0f806be7d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:18.859386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-cc54678c-f634-4ab6-b707-47f0f806be7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:18.859386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:18.859386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:18.859386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:18.859386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:18.859386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:18.859386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:18.859386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:18.859386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:18.859386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:18.859386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:18.859386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:18.859386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:18.859386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:18.859386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:18.860316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:18.860316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:18.860316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:18.860316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:18.860316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:18.860316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:18.860316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:18.860316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:18.860316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:18.860316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:18.860316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:18.860316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:18.860316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:18.860316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:18.860316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:18.860316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:18.860316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:18.861227 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:18.861227 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:18.861227 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:18.861227 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:18.861227 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:18.861227 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:18.861227 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:18.861227 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:18.861227 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 338/684] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:18.862179 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc54678c-f634-4ab6-b707-47f0f806be7d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:18.883504 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7b4e1-ef3e-4be4-8517-2403454c7032 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:18.900496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:18.931734 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f33f44-ecc8-46a4-bef2-0474b46f390c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:18.932966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f33f44-ecc8-46a4-bef2-0474b46f390c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:18.933242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f33f44-ecc8-46a4-bef2-0474b46f390c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:18.933606 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f33f44-ecc8-46a4-bef2-0474b46f390c None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:18.934568 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f33f44-ecc8-46a4-bef2-0474b46f390c None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:18.935332 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f33f44-ecc8-46a4-bef2-0474b46f390c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:18.937350 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f33f44-ecc8-46a4-bef2-0474b46f390c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:18.950064 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 347/685] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 10 16:55:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:18.951077 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f33f44-ecc8-46a4-bef2-0474b46f390c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:18.992176 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc54678c-f634-4ab6-b707-47f0f806be7d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:19.035553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:19.038467 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcbd5b9-c5d9-45a1-9a2c-1c0da3cef6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:19.039469 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbd5b9-c5d9-45a1-9a2c-1c0da3cef6ae None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:19.039859 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbd5b9-c5d9-45a1-9a2c-1c0da3cef6ae None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:19.040231 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbd5b9-c5d9-45a1-9a2c-1c0da3cef6ae None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:19.041054 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcbd5b9-c5d9-45a1-9a2c-1c0da3cef6ae None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:19.041425 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcbd5b9-c5d9-45a1-9a2c-1c0da3cef6ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:19.043729 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcbd5b9-c5d9-45a1-9a2c-1c0da3cef6ae None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:19.072853 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 339/686] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 10 16:55:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:19.072853 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcbd5b9-c5d9-45a1-9a2c-1c0da3cef6ae None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:19.099937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f33f44-ecc8-46a4-bef2-0474b46f390c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:19.152559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f117fd6-9b0b-455c-bce8-1008cf4f34a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:19.153525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f117fd6-9b0b-455c-bce8-1008cf4f34a0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:19.154257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f117fd6-9b0b-455c-bce8-1008cf4f34a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:19.155106 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f117fd6-9b0b-455c-bce8-1008cf4f34a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:19.156243 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f117fd6-9b0b-455c-bce8-1008cf4f34a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:19.156700 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f117fd6-9b0b-455c-bce8-1008cf4f34a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:19.158673 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f117fd6-9b0b-455c-bce8-1008cf4f34a0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:19.181425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 348/687] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:19.181425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f117fd6-9b0b-455c-bce8-1008cf4f34a0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:19.185923 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:19.202127 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcbd5b9-c5d9-45a1-9a2c-1c0da3cef6ae None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:19.237474 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52637ecb-6d88-4742-9aa2-f023c5477dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:19.238108 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52637ecb-6d88-4742-9aa2-f023c5477dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:19.238405 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52637ecb-6d88-4742-9aa2-f023c5477dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:19.238687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52637ecb-6d88-4742-9aa2-f023c5477dd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:19.249759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52637ecb-6d88-4742-9aa2-f023c5477dd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:19.251246 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52637ecb-6d88-4742-9aa2-f023c5477dd1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:19.266683 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-52637ecb-6d88-4742-9aa2-f023c5477dd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:19.266683 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:19.266683 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:19.266683 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:19.266683 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:19.266683 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:19.266683 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:19.266683 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:19.266683 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:19.266683 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:19.266683 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:19.266683 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:19.266683 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:19.266683 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:19.266683 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:19.266683 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:19.267871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:19.267871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:19.267871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:19.267871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:19.267871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:19.267871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:19.267871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:19.267871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:19.267871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:19.267871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:19.267871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:19.267871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:19.267871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:19.267871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:19.267871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:19.267871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:19.267871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:19.269064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:19.269064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:19.269064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:19.269064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 340/688] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:19.269064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52637ecb-6d88-4742-9aa2-f023c5477dd1 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:19.298426 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f117fd6-9b0b-455c-bce8-1008cf4f34a0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:19.324824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:19.339406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6363c8-c192-470d-b346-46fd4a646190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:19.340236 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6363c8-c192-470d-b346-46fd4a646190 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:19.340955 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6363c8-c192-470d-b346-46fd4a646190 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:19.341385 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6363c8-c192-470d-b346-46fd4a646190 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:19.375299 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52637ecb-6d88-4742-9aa2-f023c5477dd1 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:19.412054 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb345438-bf63-44b7-a115-d77b5087949f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:19.412681 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb345438-bf63-44b7-a115-d77b5087949f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:19.412958 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb345438-bf63-44b7-a115-d77b5087949f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:19.413230 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb345438-bf63-44b7-a115-d77b5087949f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:19.413955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb345438-bf63-44b7-a115-d77b5087949f None admin] RBAC: Adding query filter params (role.id=303e0fba379a41019ecc2e119d87fb1b, scope.project.id=d11f33af53374dffa61132642a7770f4, user.id=10f4f54890624da29ff217081445b6bf) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:19.414261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb345438-bf63-44b7-a115-d77b5087949f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:19.415729 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb345438-bf63-44b7-a115-d77b5087949f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:19.421693 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6363c8-c192-470d-b346-46fd4a646190 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:19.426385 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 341/689] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:55:19 2022] GET /identity/v3/role_assignments?user.id=10f4f54890624da29ff217081445b6bf&scope.project.id=d11f33af53374dffa61132642a7770f4&role.id=303e0fba379a41019ecc2e119d87fb1b => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:19.426385 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb345438-bf63-44b7-a115-d77b5087949f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:19.427442 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:19.428666 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6363c8-c192-470d-b346-46fd4a646190 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:19.570674 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 349/690] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:55:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:55:19.574373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6363c8-c192-470d-b346-46fd4a646190 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:19.602194 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:19.656726 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb345438-bf63-44b7-a115-d77b5087949f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:19.706274 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6363c8-c192-470d-b346-46fd4a646190 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:19.718574 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e80f8d-91da-44b2-86c4-c77a1adbb2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:19.719463 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e80f8d-91da-44b2-86c4-c77a1adbb2f8 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:19.719911 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e80f8d-91da-44b2-86c4-c77a1adbb2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:19.720332 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e80f8d-91da-44b2-86c4-c77a1adbb2f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:19.721138 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e80f8d-91da-44b2-86c4-c77a1adbb2f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:19.721597 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e80f8d-91da-44b2-86c4-c77a1adbb2f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:19.723148 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e80f8d-91da-44b2-86c4-c77a1adbb2f8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:19.738199 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 342/691] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:19.745247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39e9e2-f5e8-4f44-bb9a-ae63cf92d15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:19.746116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e80f8d-91da-44b2-86c4-c77a1adbb2f8 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:19.746895 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39e9e2-f5e8-4f44-bb9a-ae63cf92d15b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:19.747188 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39e9e2-f5e8-4f44-bb9a-ae63cf92d15b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:19.747458 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39e9e2-f5e8-4f44-bb9a-ae63cf92d15b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:19.758282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f39e9e2-f5e8-4f44-bb9a-ae63cf92d15b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:19.759773 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f39e9e2-f5e8-4f44-bb9a-ae63cf92d15b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:19.776229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-3f39e9e2-f5e8-4f44-bb9a-ae63cf92d15b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:19.776229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:19.776229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:19.776229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:19.776229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:19.776229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:19.776229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:19.776229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:19.776229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:19.776229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:19.776229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:19.776229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:19.776229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:19.776229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:19.776229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:19.776229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:19.777486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:19.777486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:19.777486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:19.777486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:19.777486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:19.777486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:19.777486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:19.777486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:19.777486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:19.777486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:19.777486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:19.777486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:19.777486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:19.777486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:19.777486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:19.777486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:19.777486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:19.779800 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:19.779800 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:19.779800 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:19.779800 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 350/692] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:19.783257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39e9e2-f5e8-4f44-bb9a-ae63cf92d15b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:19.827912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:19.870636 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:19.876972 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e80f8d-91da-44b2-86c4-c77a1adbb2f8 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:19.898056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39e9e2-f5e8-4f44-bb9a-ae63cf92d15b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:19.916551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0112f-bcf6-4ba7-ae18-f5997f793288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:19.917570 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0112f-bcf6-4ba7-ae18-f5997f793288 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:19.918161 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0112f-bcf6-4ba7-ae18-f5997f793288 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:19.918673 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0112f-bcf6-4ba7-ae18-f5997f793288 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:19.927695 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0112f-bcf6-4ba7-ae18-f5997f793288 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:19.929551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0112f-bcf6-4ba7-ae18-f5997f793288 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:19.939446 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58df1d2-1fd9-405f-8a21-d3d38e489cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:19.940059 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58df1d2-1fd9-405f-8a21-d3d38e489cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:19.940338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58df1d2-1fd9-405f-8a21-d3d38e489cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:19.940610 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58df1d2-1fd9-405f-8a21-d3d38e489cc6 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:19.941257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58df1d2-1fd9-405f-8a21-d3d38e489cc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:19.941558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58df1d2-1fd9-405f-8a21-d3d38e489cc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:19.942977 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58df1d2-1fd9-405f-8a21-d3d38e489cc6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:19.944733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-aeb0112f-bcf6-4ba7-ae18-f5997f793288 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 16:55:19.944733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:19.944733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:19.944733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:19.944733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:19.944733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:19.944733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:19.944733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:19.944733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:19.944733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:19.944733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:19.944733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:19.944733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:19.944733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:19.944733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:19.944733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:19.945661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:19.945661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:19.945661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:19.945661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:19.945661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:19.945661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:19.945661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:19.945661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:19.945661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:19.945661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:19.945661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:19.945661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:19.945661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:19.945661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:19.945661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:19.945661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:19.945661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:19.945661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:19.946648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:19.946648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:19.946648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:19.946648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:19.946648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:19.946648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:19.946648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:19.946648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:19.946648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 16:55:19.946648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:19.951386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 343/693] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:55:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:19.956715 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 351/694] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:19.958035 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0112f-bcf6-4ba7-ae18-f5997f793288 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:19.962032 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58df1d2-1fd9-405f-8a21-d3d38e489cc6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:20.067838 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58df1d2-1fd9-405f-8a21-d3d38e489cc6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:20.072592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0112f-bcf6-4ba7-ae18-f5997f793288 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:20.099809 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:20.105758 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6320d8-676f-493f-9465-671445d5d602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:20.106427 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6320d8-676f-493f-9465-671445d5d602 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:20.106709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6320d8-676f-493f-9465-671445d5d602 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:20.107003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6320d8-676f-493f-9465-671445d5d602 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:20.109578 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:20.115266 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6320d8-676f-493f-9465-671445d5d602 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:20.116152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cac256-f81d-4fbe-ae0e-7d21da928a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:20.116788 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6320d8-676f-493f-9465-671445d5d602 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:20.117427 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cac256-f81d-4fbe-ae0e-7d21da928a99 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:20.117918 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cac256-f81d-4fbe-ae0e-7d21da928a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:20.118420 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cac256-f81d-4fbe-ae0e-7d21da928a99 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:20.119347 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cac256-f81d-4fbe-ae0e-7d21da928a99 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:20.119862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cac256-f81d-4fbe-ae0e-7d21da928a99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:20.122420 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cac256-f81d-4fbe-ae0e-7d21da928a99 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:20.129290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ff6320d8-676f-493f-9465-671445d5d602 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:20.129290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:20.129290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:20.129290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:20.129290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:20.129290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:20.129290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:20.129290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:20.129290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:20.129290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:20.129290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:20.129290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:20.129290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:20.129290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:20.129290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:20.130228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:20.130228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:20.130228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:20.130228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:20.130228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:20.130228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:20.130228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:20.130228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:20.130228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:20.130228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:20.130228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:20.130228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:20.130228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:20.130228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:20.130228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:20.130228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:20.130228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:20.131180 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:20.131180 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:20.131180 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:20.131180 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:20.131180 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:20.131180 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:20.131180 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:20.131180 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:20.131180 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 352/695] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:55:20.132185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9d639-8a67-4da2-9caf-e87702645ff6 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:20.132185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9d639-8a67-4da2-9caf-e87702645ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:20.132437 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9d639-8a67-4da2-9caf-e87702645ff6 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:20.133782 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 353/696] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:20.139011 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9d639-8a67-4da2-9caf-e87702645ff6 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:20.142151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 344/697] 10.222.0.45 () {56 vars in 1135 bytes} [Wed Aug 10 16:55:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:20.143460 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73340fa-67ee-46d8-b403-b508f62e4c73 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:20.143736 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73340fa-67ee-46d8-b403-b508f62e4c73 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:20.144001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73340fa-67ee-46d8-b403-b508f62e4c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:20.167431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b73340fa-67ee-46d8-b403-b508f62e4c73 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:20.168680 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b73340fa-67ee-46d8-b403-b508f62e4c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:20.242876 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 345/698] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:55:20.243887 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73340fa-67ee-46d8-b403-b508f62e4c73 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:20.259077 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9d639-8a67-4da2-9caf-e87702645ff6 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:20.296699 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6181cf7-b6ae-4bfd-9b70-eb5e4a179999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:20.297311 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6181cf7-b6ae-4bfd-9b70-eb5e4a179999 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:20.297589 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6181cf7-b6ae-4bfd-9b70-eb5e4a179999 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:20.297865 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6181cf7-b6ae-4bfd-9b70-eb5e4a179999 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:20.298585 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6181cf7-b6ae-4bfd-9b70-eb5e4a179999 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:20.298890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6181cf7-b6ae-4bfd-9b70-eb5e4a179999 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:20.300333 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6181cf7-b6ae-4bfd-9b70-eb5e4a179999 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:20.314088 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 354/699] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:20.315048 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6181cf7-b6ae-4bfd-9b70-eb5e4a179999 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:20.353394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73340fa-67ee-46d8-b403-b508f62e4c73 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:20.366633 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:20.379927 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:20.394042 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0622da2-5acc-4697-abf7-7d93dd5df6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:20.394832 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0622da2-5acc-4697-abf7-7d93dd5df6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:20.395211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0622da2-5acc-4697-abf7-7d93dd5df6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:20.395545 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0622da2-5acc-4697-abf7-7d93dd5df6a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:20.431499 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6181cf7-b6ae-4bfd-9b70-eb5e4a179999 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:20.472227 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4919e-20ce-448a-91a2-8ecf72c3c89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:20.473153 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4919e-20ce-448a-91a2-8ecf72c3c89a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:20.473275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0622da2-5acc-4697-abf7-7d93dd5df6a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:20.473703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4919e-20ce-448a-91a2-8ecf72c3c89a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:20.474123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4919e-20ce-448a-91a2-8ecf72c3c89a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:20.474890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4919e-20ce-448a-91a2-8ecf72c3c89a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:20.475005 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0622da2-5acc-4697-abf7-7d93dd5df6a0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:20.476501 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4919e-20ce-448a-91a2-8ecf72c3c89a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:20.489416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-fba4919e-20ce-448a-91a2-8ecf72c3c89a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 16:55:20.489416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:20.489416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:55:20.489416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:55:20.489416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:55:20.489416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:55:20.489416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:55:20.489416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:20.489416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 16:55:20.489416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:20.489416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:55:20.489416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:20.489416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:20.489416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:20.489416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:20.489416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:20.489416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:20.490763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:20.490763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:20.490763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:20.490763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:20.490763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:20.490763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:20.490763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:55:20.490763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:55:20.490763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:55:20.490763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:55:20.490763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:20.490763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:20.490763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:20.490763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:20.490763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:20.490763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:20.491918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:20.491918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:20.491918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:20.491918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:20.491918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:20.491918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:20.491918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:20.491918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:20.491918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:20.491918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:20.491918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:55:20.491918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:20.491918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 16:55:20.491918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:20.494578 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 355/700] 10.222.0.45 () {56 vars in 1129 bytes} [Wed Aug 10 16:55:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:20.496265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4919e-20ce-448a-91a2-8ecf72c3c89a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:20.599798 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 346/701] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:55:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 16:55:20.600994 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0622da2-5acc-4697-abf7-7d93dd5df6a0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:20.611082 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4919e-20ce-448a-91a2-8ecf72c3c89a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:20.625581 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:20.631469 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:20.650516 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8124002-736a-4bef-b2f4-2ddb73affcd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:20.652052 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8124002-736a-4bef-b2f4-2ddb73affcd1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:20.652542 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8124002-736a-4bef-b2f4-2ddb73affcd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:20.653039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8124002-736a-4bef-b2f4-2ddb73affcd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:20.654009 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8124002-736a-4bef-b2f4-2ddb73affcd1 None admin] RBAC: Adding query filter params (role.id=7554b353873942fab506de830188aba0, scope.project.id=d11f33af53374dffa61132642a7770f4, user.id=615cc36e21cc493b823e6cee695ac308) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:20.654556 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8124002-736a-4bef-b2f4-2ddb73affcd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:20.656238 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8124002-736a-4bef-b2f4-2ddb73affcd1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:20.662649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 356/702] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:55:20 2022] GET /identity/v3/role_assignments?user.id=615cc36e21cc493b823e6cee695ac308&scope.project.id=d11f33af53374dffa61132642a7770f4&role.id=7554b353873942fab506de830188aba0 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:20.663755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8124002-736a-4bef-b2f4-2ddb73affcd1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:20.785032 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0622da2-5acc-4697-abf7-7d93dd5df6a0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:20.847332 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8124002-736a-4bef-b2f4-2ddb73affcd1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:20.858273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d470755-1cd9-45ed-959e-7f06f19af6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:20.859737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d470755-1cd9-45ed-959e-7f06f19af6f6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:20.860490 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d470755-1cd9-45ed-959e-7f06f19af6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:20.860968 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d470755-1cd9-45ed-959e-7f06f19af6f6 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:20.861986 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d470755-1cd9-45ed-959e-7f06f19af6f6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:20.862928 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d470755-1cd9-45ed-959e-7f06f19af6f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:20.865754 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d470755-1cd9-45ed-959e-7f06f19af6f6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:20.873817 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 347/703] 10.222.0.45 () {56 vars in 1124 bytes} [Wed Aug 10 16:55:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:20.878814 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d470755-1cd9-45ed-959e-7f06f19af6f6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:20.893633 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5b2c3-3b55-4ffd-9441-5bf88100015d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:20.894269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5b2c3-3b55-4ffd-9441-5bf88100015d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:20.894582 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5b2c3-3b55-4ffd-9441-5bf88100015d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:20.894855 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5b2c3-3b55-4ffd-9441-5bf88100015d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:20.907992 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc5b2c3-3b55-4ffd-9441-5bf88100015d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:20.909465 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc5b2c3-3b55-4ffd-9441-5bf88100015d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:20.926920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-cdc5b2c3-3b55-4ffd-9441-5bf88100015d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:20.926920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:20.926920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:20.926920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:20.926920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:20.926920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:20.926920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:20.926920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:20.926920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:20.926920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:20.926920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:20.926920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:20.926920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:20.926920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:20.926920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:20.926920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:20.927966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:20.927966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:20.927966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:20.927966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:20.927966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:20.927966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:20.927966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:20.927966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:20.927966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:20.927966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:20.927966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:20.927966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:20.927966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:20.927966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:20.927966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:20.927966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:20.927966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:20.928921 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:20.928921 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:20.928921 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:20.928921 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 357/704] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:20.929778 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ff771-eec6-4439-9f9a-6b53fe59cc71 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:20.930062 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ff771-eec6-4439-9f9a-6b53fe59cc71 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:20.930336 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ff771-eec6-4439-9f9a-6b53fe59cc71 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:20.936505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 358/705] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:20.936505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ff771-eec6-4439-9f9a-6b53fe59cc71 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:20.978037 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:20.985963 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:21.013665 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d470755-1cd9-45ed-959e-7f06f19af6f6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:21.052317 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd3723-4ce3-465e-8bd9-eb1bfbcb6292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:21.053080 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd3723-4ce3-465e-8bd9-eb1bfbcb6292 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:21.053356 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ff771-eec6-4439-9f9a-6b53fe59cc71 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:21.053545 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd3723-4ce3-465e-8bd9-eb1bfbcb6292 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:21.053918 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd3723-4ce3-465e-8bd9-eb1bfbcb6292 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:21.054935 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd3723-4ce3-465e-8bd9-eb1bfbcb6292 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:21.055324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd3723-4ce3-465e-8bd9-eb1bfbcb6292 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:21.058704 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd3723-4ce3-465e-8bd9-eb1bfbcb6292 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:21.065574 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 348/706] 10.222.0.45 () {56 vars in 1124 bytes} [Wed Aug 10 16:55:20 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:21.067452 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303ce9a-6422-4d9c-9b70-b4104182b97a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:21.067892 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303ce9a-6422-4d9c-9b70-b4104182b97a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:21.068266 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303ce9a-6422-4d9c-9b70-b4104182b97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:21.087859 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8303ce9a-6422-4d9c-9b70-b4104182b97a None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:21.089512 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8303ce9a-6422-4d9c-9b70-b4104182b97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:21.098168 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c32cec-debf-4f0f-88e4-bf25b289b8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:21.105106 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c32cec-debf-4f0f-88e4-bf25b289b8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:21.105106 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c32cec-debf-4f0f-88e4-bf25b289b8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:21.105106 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c32cec-debf-4f0f-88e4-bf25b289b8e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:21.105106 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c32cec-debf-4f0f-88e4-bf25b289b8e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:21.105106 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c32cec-debf-4f0f-88e4-bf25b289b8e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:21.105106 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c32cec-debf-4f0f-88e4-bf25b289b8e6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:21.118583 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 359/707] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:21.130048 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c32cec-debf-4f0f-88e4-bf25b289b8e6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:21.209186 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 349/708] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:55:21.258431 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:21.266825 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c32cec-debf-4f0f-88e4-bf25b289b8e6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:21.283607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303ce9a-6422-4d9c-9b70-b4104182b97a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:21.311041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79382b1a-0484-43a2-af63-d3d5d1c4c701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:21.312066 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79382b1a-0484-43a2-af63-d3d5d1c4c701 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:21.312578 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79382b1a-0484-43a2-af63-d3d5d1c4c701 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:21.313053 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79382b1a-0484-43a2-af63-d3d5d1c4c701 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:21.320611 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79382b1a-0484-43a2-af63-d3d5d1c4c701 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:21.322369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79382b1a-0484-43a2-af63-d3d5d1c4c701 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:21.334421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-79382b1a-0484-43a2-af63-d3d5d1c4c701 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:21.334421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:21.334421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:21.334421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:21.334421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:21.334421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:21.334421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:21.334421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:21.334421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:21.334421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:21.334421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:21.334421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:21.334421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:21.334421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:21.334421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:21.335342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:21.335342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:21.335342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:21.335342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:21.335342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:21.335342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:21.335342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:21.335342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:21.335342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:21.335342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:21.335342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:21.335342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:21.335342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:21.335342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:21.335342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:21.335342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:21.335342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:21.336224 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:21.336224 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:21.336224 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:21.336224 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:21.336224 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:21.336224 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:21.336224 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:21.336224 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:21.338590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 360/709] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:21.345112 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79382b1a-0484-43a2-af63-d3d5d1c4c701 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:21.377257 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:21.414962 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303ce9a-6422-4d9c-9b70-b4104182b97a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:21.455970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e186b7-0367-4017-a374-177c33968401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:21.456976 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e186b7-0367-4017-a374-177c33968401 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:21.457460 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e186b7-0367-4017-a374-177c33968401 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:21.457964 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e186b7-0367-4017-a374-177c33968401 None admin] PATH_INFO: `/v3/services/network` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:21.458806 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e186b7-0367-4017-a374-177c33968401 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:21.460392 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e186b7-0367-4017-a374-177c33968401 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:21.461560 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79382b1a-0484-43a2-af63-d3d5d1c4c701 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:21.472699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-30e186b7-0367-4017-a374-177c33968401 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 16:55:21.472699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:21.472699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:55:21.472699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:55:21.472699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:55:21.472699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:55:21.472699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:55:21.472699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:21.472699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 16:55:21.472699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:21.472699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:55:21.472699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:21.472699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:21.472699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:21.472699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:21.472699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:21.472699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:21.473748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:21.473748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:21.473748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:21.473748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:21.473748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:21.473748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:21.473748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:55:21.473748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:55:21.473748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:55:21.473748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:55:21.473748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:21.473748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:21.473748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:21.473748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:21.473748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:21.473748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:21.475046 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:21.475046 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:21.475046 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:21.475046 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:21.475046 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:21.475046 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:21.475046 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:21.475046 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:21.475046 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:21.475046 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:21.475046 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:55:21.475046 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:21.475046 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 16:55:21.475046 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:21.477530 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 350/710] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:21 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:21.484276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e186b7-0367-4017-a374-177c33968401 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:21.503654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35474234-7b40-4bf6-81ab-6d83b54dfd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:21.504271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35474234-7b40-4bf6-81ab-6d83b54dfd37 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:21.504550 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35474234-7b40-4bf6-81ab-6d83b54dfd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:21.504886 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35474234-7b40-4bf6-81ab-6d83b54dfd37 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:21.505585 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35474234-7b40-4bf6-81ab-6d83b54dfd37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:21.505889 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35474234-7b40-4bf6-81ab-6d83b54dfd37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:21.507553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35474234-7b40-4bf6-81ab-6d83b54dfd37 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:21.525023 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 361/711] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:21.527910 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35474234-7b40-4bf6-81ab-6d83b54dfd37 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:21.546737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:21.597599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e186b7-0367-4017-a374-177c33968401 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:21.630915 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:21.637127 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aff186-256f-4d05-b358-8a82ca55adb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:21.638222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aff186-256f-4d05-b358-8a82ca55adb0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:21.638772 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aff186-256f-4d05-b358-8a82ca55adb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:21.639114 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aff186-256f-4d05-b358-8a82ca55adb0 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:21.639841 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aff186-256f-4d05-b358-8a82ca55adb0 None admin] RBAC: Adding query filter params (name=network) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:21.640216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aff186-256f-4d05-b358-8a82ca55adb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:21.641716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aff186-256f-4d05-b358-8a82ca55adb0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:21.646824 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 351/712] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 10 16:55:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:21.648483 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35474234-7b40-4bf6-81ab-6d83b54dfd37 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:21.653324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aff186-256f-4d05-b358-8a82ca55adb0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:21.696703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f3c2f-939c-4e53-9dbd-68e06f3c5794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:21.697727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f3c2f-939c-4e53-9dbd-68e06f3c5794 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:21.698914 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f3c2f-939c-4e53-9dbd-68e06f3c5794 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:21.699423 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f3c2f-939c-4e53-9dbd-68e06f3c5794 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:21.700480 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f3c2f-939c-4e53-9dbd-68e06f3c5794 None admin] RBAC: Adding query filter params (role.id=303e0fba379a41019ecc2e119d87fb1b, scope.project.id=d11f33af53374dffa61132642a7770f4, user.id=4986821ef9f84b99a835e387635aa9a8) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:21.700997 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f3c2f-939c-4e53-9dbd-68e06f3c5794 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:21.702776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f3c2f-939c-4e53-9dbd-68e06f3c5794 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:21.714829 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 362/713] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:55:21 2022] GET /identity/v3/role_assignments?user.id=4986821ef9f84b99a835e387635aa9a8&scope.project.id=d11f33af53374dffa61132642a7770f4&role.id=303e0fba379a41019ecc2e119d87fb1b => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:21.768166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aff186-256f-4d05-b358-8a82ca55adb0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:21.809101 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00894a1-5e6e-40ab-aaea-d68d798c4ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:21.809730 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00894a1-5e6e-40ab-aaea-d68d798c4ffa None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:21.810006 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00894a1-5e6e-40ab-aaea-d68d798c4ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:21.810276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00894a1-5e6e-40ab-aaea-d68d798c4ffa None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:21.810975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00894a1-5e6e-40ab-aaea-d68d798c4ffa None admin] RBAC: Adding query filter params (type=network) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:21.811280 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00894a1-5e6e-40ab-aaea-d68d798c4ffa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:21.812762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00894a1-5e6e-40ab-aaea-d68d798c4ffa None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:21.818733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 352/714] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 10 16:55:21 2022] GET /identity/v3/services?type=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:22.241839 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416a307-8f5e-4eb5-984c-10e9d50ce395 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:22.242424 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416a307-8f5e-4eb5-984c-10e9d50ce395 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:22.242759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416a307-8f5e-4eb5-984c-10e9d50ce395 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:22.244684 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 363/715] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:22.250967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb52f6-a47f-4116-b7d3-c98f124c73e6 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:22.251382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb52f6-a47f-4116-b7d3-c98f124c73e6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:22.251582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb52f6-a47f-4116-b7d3-c98f124c73e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:22.271238 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-48bb52f6-a47f-4116-b7d3-c98f124c73e6 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:22.272431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-48bb52f6-a47f-4116-b7d3-c98f124c73e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:22.334061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:22.348286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 353/716] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:55:22.372283 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416a307-8f5e-4eb5-984c-10e9d50ce395 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:22.457788 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:22.481642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416a307-8f5e-4eb5-984c-10e9d50ce395 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:22.516914 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0494f9aa-2a51-4c78-a819-beb0c976ab73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:22.517477 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494f9aa-2a51-4c78-a819-beb0c976ab73 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:22.517761 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494f9aa-2a51-4c78-a819-beb0c976ab73 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:22.518040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494f9aa-2a51-4c78-a819-beb0c976ab73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:22.530808 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0494f9aa-2a51-4c78-a819-beb0c976ab73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:22.532354 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0494f9aa-2a51-4c78-a819-beb0c976ab73 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:22.547089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0494f9aa-2a51-4c78-a819-beb0c976ab73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:22.547089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:22.547089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:22.547089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:22.547089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:22.547089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:22.547089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:22.547089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:22.547089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:22.547089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:22.547089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:22.547089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:22.547089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:22.547089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:22.547089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:22.547089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:22.548051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:22.548051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:22.548051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:22.548051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:22.548051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:22.548051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:22.548051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:22.548051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:22.548051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:22.548051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:22.548051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:22.548051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:22.548051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:22.548051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:22.548051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:22.548051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:22.548051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:22.548934 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:22.548934 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:22.548934 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:22.553119 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 364/717] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:22.554527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb52f6-a47f-4116-b7d3-c98f124c73e6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:22.696394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb52f6-a47f-4116-b7d3-c98f124c73e6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:22.754327 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa85f0-750f-4166-aa83-0b93b7fab147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:22.754966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa85f0-750f-4166-aa83-0b93b7fab147 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:22.755244 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa85f0-750f-4166-aa83-0b93b7fab147 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:22.755507 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa85f0-750f-4166-aa83-0b93b7fab147 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:22.756155 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfa85f0-750f-4166-aa83-0b93b7fab147 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:22.756514 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfa85f0-750f-4166-aa83-0b93b7fab147 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:22.757932 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfa85f0-750f-4166-aa83-0b93b7fab147 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:22.773891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 354/718] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:22.780018 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0494f9aa-2a51-4c78-a819-beb0c976ab73 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:22.919563 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0494f9aa-2a51-4c78-a819-beb0c976ab73 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:22.962763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:22.967354 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88a8c3-180a-4957-b68e-0e6ea05e6589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:22.968027 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88a8c3-180a-4957-b68e-0e6ea05e6589 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:22.968308 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88a8c3-180a-4957-b68e-0e6ea05e6589 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:22.968628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88a8c3-180a-4957-b68e-0e6ea05e6589 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:22.969282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88a8c3-180a-4957-b68e-0e6ea05e6589 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:22.970759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88a8c3-180a-4957-b68e-0e6ea05e6589 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:23.056924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 365/719] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 10 16:55:22 2022] POST /identity/v3/users => generated 254 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:23.150253 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e1fd0-99b6-4b76-8034-b088b9efce7a None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:23.150626 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e1fd0-99b6-4b76-8034-b088b9efce7a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:23.150828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e1fd0-99b6-4b76-8034-b088b9efce7a None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:23.152660 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 355/720] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:23.158310 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ead22-4b0a-42ae-ab56-5eddddf5079a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:23.158672 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ead22-4b0a-42ae-ab56-5eddddf5079a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:23.158905 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ead22-4b0a-42ae-ab56-5eddddf5079a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:23.189071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ec4ead22-4b0a-42ae-ab56-5eddddf5079a None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:23.190293 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ec4ead22-4b0a-42ae-ab56-5eddddf5079a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:23.287883 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 366/721] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:55:23.320538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e1fd0-99b6-4b76-8034-b088b9efce7a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:23.375783 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:23.428630 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e1fd0-99b6-4b76-8034-b088b9efce7a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:23.464707 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e60601-629e-4697-aaf5-6430a392cb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:23.465377 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e60601-629e-4697-aaf5-6430a392cb17 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:23.465657 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e60601-629e-4697-aaf5-6430a392cb17 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:23.465927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e60601-629e-4697-aaf5-6430a392cb17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:23.476597 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e60601-629e-4697-aaf5-6430a392cb17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:23.482662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-e9e60601-629e-4697-aaf5-6430a392cb17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:55:23.482662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:23.482662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:23.482662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:23.482662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:23.482662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:23.482662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:23.482662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:23.482662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:23.482662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:23.482662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:23.482662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:23.482662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:23.482662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:23.482662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:23.482662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:23.483675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:23.483675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:23.483675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:23.483675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:23.483675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:23.483675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:23.483675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:23.483675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:23.483675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:23.483675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:23.483675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:23.483675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:23.483675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:23.483675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:23.483675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:23.483675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:23.484551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:23.484551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:23.484551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:23.484551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:23.484551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:23.484551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:23.484551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:23.484551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:23.484551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:23.484551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:23.484551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:55:23.484551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:23.484551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 356/722] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:55:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:23.490451 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ead22-4b0a-42ae-ab56-5eddddf5079a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:23.601595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ead22-4b0a-42ae-ab56-5eddddf5079a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:23.634836 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:23.660655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d89f2a-f9a6-4638-9077-554aed3a65f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:23.661644 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d89f2a-f9a6-4638-9077-554aed3a65f0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:23.662151 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d89f2a-f9a6-4638-9077-554aed3a65f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:23.662652 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d89f2a-f9a6-4638-9077-554aed3a65f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:23.663527 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d89f2a-f9a6-4638-9077-554aed3a65f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:23.664081 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d89f2a-f9a6-4638-9077-554aed3a65f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:23.665643 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d89f2a-f9a6-4638-9077-554aed3a65f0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:23.677092 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 367/723] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:55:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:23.682470 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e60601-629e-4697-aaf5-6430a392cb17 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:23.770904 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f77da-f202-426c-9b31-aaebc6462c64 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:23.771235 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f77da-f202-426c-9b31-aaebc6462c64 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:23.771455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f77da-f202-426c-9b31-aaebc6462c64 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:23.772803 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 368/724] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:55:23.787957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f650d-61cb-46e8-9d10-2b06e5762132 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:23.788263 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f650d-61cb-46e8-9d10-2b06e5762132 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:23.788505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f650d-61cb-46e8-9d10-2b06e5762132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:23.826369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-fa0f650d-61cb-46e8-9d10-2b06e5762132 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:23.827573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fa0f650d-61cb-46e8-9d10-2b06e5762132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:23.840580 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e60601-629e-4697-aaf5-6430a392cb17 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:23.867202 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:23.906832 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1432c1e-89d6-4666-b338-69a488634fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:23.907477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1432c1e-89d6-4666-b338-69a488634fbc None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:23.907761 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1432c1e-89d6-4666-b338-69a488634fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:23.908037 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1432c1e-89d6-4666-b338-69a488634fbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:23.929811 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1432c1e-89d6-4666-b338-69a488634fbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:23.931866 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1432c1e-89d6-4666-b338-69a488634fbc None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:23.958374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-b1432c1e-89d6-4666-b338-69a488634fbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:23.958374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:23.958374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:23.958374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:23.958374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:23.958374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:23.958374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:23.958374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:23.958374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:23.958374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:23.958374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:23.958374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:23.958374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:23.958374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:23.958374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:23.958374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:23.959375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:23.959375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:23.959375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:23.959375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:23.959375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:23.959375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:23.959375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:23.959375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:23.959375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:23.959375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:23.959375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:23.959375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:23.959375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:23.959375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:23.959375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:23.959375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:23.959375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:23.960247 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:23.960247 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:23.960247 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:23.960247 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 357/725] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:23.960247 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1432c1e-89d6-4666-b338-69a488634fbc None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:23.972647 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 369/726] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 16:55:23.999376 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f650d-61cb-46e8-9d10-2b06e5762132 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:24.071508 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1432c1e-89d6-4666-b338-69a488634fbc None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:24.108109 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9a4ae-c658-4f15-a7c0-4ce8049ec054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:24.108691 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9a4ae-c658-4f15-a7c0-4ce8049ec054 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:24.108968 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9a4ae-c658-4f15-a7c0-4ce8049ec054 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:24.109239 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9a4ae-c658-4f15-a7c0-4ce8049ec054 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:24.109886 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f9a4ae-c658-4f15-a7c0-4ce8049ec054 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:24.110187 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f9a4ae-c658-4f15-a7c0-4ce8049ec054 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:24.111596 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f9a4ae-c658-4f15-a7c0-4ce8049ec054 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:24.125096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 358/727] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:24.130579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9a4ae-c658-4f15-a7c0-4ce8049ec054 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:24.159377 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:24.232288 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f650d-61cb-46e8-9d10-2b06e5762132 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:24.237344 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9a4ae-c658-4f15-a7c0-4ce8049ec054 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:24.242593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:24.269144 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061cd76a-dfe3-42bd-bb3b-6ed0a15c6562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:24.270109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cd76a-dfe3-42bd-bb3b-6ed0a15c6562 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:24.270640 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cd76a-dfe3-42bd-bb3b-6ed0a15c6562 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:24.271106 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cd76a-dfe3-42bd-bb3b-6ed0a15c6562 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:24.271954 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061cd76a-dfe3-42bd-bb3b-6ed0a15c6562 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:24.273508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061cd76a-dfe3-42bd-bb3b-6ed0a15c6562 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:24.279734 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22285f-2804-4f18-8869-1958904d56f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:24.280402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22285f-2804-4f18-8869-1958904d56f3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:24.280674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22285f-2804-4f18-8869-1958904d56f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:24.280947 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22285f-2804-4f18-8869-1958904d56f3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:24.289730 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22285f-2804-4f18-8869-1958904d56f3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:24.291203 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22285f-2804-4f18-8869-1958904d56f3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:24.305953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-ed22285f-2804-4f18-8869-1958904d56f3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:55:24.305953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:24.305953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:24.305953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:24.305953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:24.305953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:24.305953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:24.305953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:24.305953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:24.305953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:24.305953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:24.305953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:24.305953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:24.305953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:24.305953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:24.305953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:24.307021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:24.307021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:24.307021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:24.307021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:24.307021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:24.307021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:24.307021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:24.307021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:24.307021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:24.307021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:24.307021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:24.307021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:24.307021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:24.307021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:24.307021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:24.307021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:24.307021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:24.307021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:24.309233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:24.309233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:24.309233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:24.309233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:24.309233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:24.309233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:24.309233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:24.309233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:24.309233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:55:24.309233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:24.309233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 359/728] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 10 16:55:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:24.309874 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08184830-0c46-44bd-8cdf-941bffc4681f None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:24.309874 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08184830-0c46-44bd-8cdf-941bffc4681f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:24.310030 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08184830-0c46-44bd-8cdf-941bffc4681f None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:24.316546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 360/729] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:24.316546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08184830-0c46-44bd-8cdf-941bffc4681f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:24.409077 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:24.412007 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 370/730] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 10 16:55:23 2022] POST /identity/v3/services => generated 251 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:55:24.414384 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb6771-6f7f-4107-90fb-55ad2ffcb453 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:24.414996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb6771-6f7f-4107-90fb-55ad2ffcb453 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:24.416081 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb6771-6f7f-4107-90fb-55ad2ffcb453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:24.441259 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9efb6771-6f7f-4107-90fb-55ad2ffcb453 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:24.444000 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9efb6771-6f7f-4107-90fb-55ad2ffcb453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:24.444872 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08184830-0c46-44bd-8cdf-941bffc4681f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:24.486617 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94436c-e479-49c5-ac2b-03ef3d44b422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:24.487386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94436c-e479-49c5-ac2b-03ef3d44b422 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:24.487731 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94436c-e479-49c5-ac2b-03ef3d44b422 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:24.488098 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94436c-e479-49c5-ac2b-03ef3d44b422 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:24.489247 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94436c-e479-49c5-ac2b-03ef3d44b422 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:24.489370 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94436c-e479-49c5-ac2b-03ef3d44b422 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:24.490933 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94436c-e479-49c5-ac2b-03ef3d44b422 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:24.526817 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 361/731] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 10 16:55:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:24.532426 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94436c-e479-49c5-ac2b-03ef3d44b422 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:24.541677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 371/732] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:24 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 16:55:24.566170 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb6771-6f7f-4107-90fb-55ad2ffcb453 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:24.688719 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:24.695435 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb6771-6f7f-4107-90fb-55ad2ffcb453 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:24.735899 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ab580-04d6-4840-8c4f-915f1b8e517b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:24.736497 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ab580-04d6-4840-8c4f-915f1b8e517b None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:24.736778 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ab580-04d6-4840-8c4f-915f1b8e517b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:24.737051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ab580-04d6-4840-8c4f-915f1b8e517b None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:24.737688 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ab580-04d6-4840-8c4f-915f1b8e517b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:24.738985 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94436c-e479-49c5-ac2b-03ef3d44b422 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:24.739107 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ab580-04d6-4840-8c4f-915f1b8e517b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:24.781377 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48824080-b13b-4d2d-8945-610c04a239b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:24.782462 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48824080-b13b-4d2d-8945-610c04a239b6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:24.782462 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48824080-b13b-4d2d-8945-610c04a239b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:24.782632 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48824080-b13b-4d2d-8945-610c04a239b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:24.785431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:24.794056 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48824080-b13b-4d2d-8945-610c04a239b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:24.795637 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48824080-b13b-4d2d-8945-610c04a239b6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:24.810382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-48824080-b13b-4d2d-8945-610c04a239b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:24.810382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:24.810382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:24.810382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:24.810382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:24.810382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:24.810382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:24.810382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:24.810382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:24.810382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:24.810382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:24.810382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:24.810382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:24.810382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:24.810382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:24.810382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:24.811427 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:24.811427 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:24.811427 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:24.811427 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:24.811427 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:24.811427 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:24.811427 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:24.811427 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:24.811427 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:24.811427 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:24.811427 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:24.811427 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:24.811427 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:24.811427 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:24.811427 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:24.811427 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:24.811427 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:24.812341 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:24.812341 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:24.812341 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:24.812341 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 362/733] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:24.813024 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4534f8-57c1-43c5-9ab5-ec81dd0a3759 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:24.814284 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4534f8-57c1-43c5-9ab5-ec81dd0a3759 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:24.814284 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4534f8-57c1-43c5-9ab5-ec81dd0a3759 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:24.815089 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 363/734] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:24.820985 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a713a58-9434-4ebc-ac14-6dfd0b1916f0 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:24.821454 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a713a58-9434-4ebc-ac14-6dfd0b1916f0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:24.821959 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a713a58-9434-4ebc-ac14-6dfd0b1916f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:24.833448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 372/735] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 10 16:55:24 2022] POST /identity/v3/services => generated 240 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:24.834411 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ab580-04d6-4840-8c4f-915f1b8e517b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:24.864019 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0a713a58-9434-4ebc-ac14-6dfd0b1916f0 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:24.865249 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0a713a58-9434-4ebc-ac14-6dfd0b1916f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:25.000670 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 364/736] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:55:25.019238 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ab580-04d6-4840-8c4f-915f1b8e517b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:25.027184 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a713a58-9434-4ebc-ac14-6dfd0b1916f0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:25.058540 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed8464-cce0-4531-b33c-c8a4ecae7066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:25.059640 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed8464-cce0-4531-b33c-c8a4ecae7066 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:25.060055 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed8464-cce0-4531-b33c-c8a4ecae7066 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:25.060430 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed8464-cce0-4531-b33c-c8a4ecae7066 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:25.061242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed8464-cce0-4531-b33c-c8a4ecae7066 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:25.061680 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed8464-cce0-4531-b33c-c8a4ecae7066 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:25.063460 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed8464-cce0-4531-b33c-c8a4ecae7066 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:25.083853 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 373/737] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:25.085312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed8464-cce0-4531-b33c-c8a4ecae7066 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:25.109964 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:25.188562 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a713a58-9434-4ebc-ac14-6dfd0b1916f0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:25.206020 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:25.225167 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed8464-cce0-4531-b33c-c8a4ecae7066 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:25.231372 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3bf1e6-4f3a-4a25-8df7-361a5070f579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:25.232008 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3bf1e6-4f3a-4a25-8df7-361a5070f579 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:25.232282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3bf1e6-4f3a-4a25-8df7-361a5070f579 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:25.232548 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3bf1e6-4f3a-4a25-8df7-361a5070f579 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:25.243851 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3bf1e6-4f3a-4a25-8df7-361a5070f579 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:25.249509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-6c3bf1e6-4f3a-4a25-8df7-361a5070f579 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:55:25.249509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:25.249509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:25.249509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:25.249509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:25.249509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:25.249509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:25.249509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:25.249509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:25.249509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:25.249509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:25.249509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:25.249509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:25.249509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:25.249509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:25.249509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:25.250509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:25.250509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:25.250509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:25.250509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:25.250509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:25.250509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:25.250509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:25.250509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:25.250509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:25.250509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:25.250509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:25.250509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:25.250509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:25.250509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:25.250509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:25.250509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:25.251743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:25.251743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:25.251743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:25.251743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:25.251743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:25.251743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:25.251743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:25.251743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:25.251743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:25.251743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:25.251743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:55:25.251743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:25.251743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 365/738] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:55:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:55:25.256872 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3bf1e6-4f3a-4a25-8df7-361a5070f579 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:25.273494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b988232-9fc7-4721-8bc3-4de2d6e917d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:25.274722 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b988232-9fc7-4721-8bc3-4de2d6e917d3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:25.275452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b988232-9fc7-4721-8bc3-4de2d6e917d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:25.275889 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b988232-9fc7-4721-8bc3-4de2d6e917d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:25.285527 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b988232-9fc7-4721-8bc3-4de2d6e917d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:25.287681 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b988232-9fc7-4721-8bc3-4de2d6e917d3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:25.303452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2b988232-9fc7-4721-8bc3-4de2d6e917d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:25.303452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:25.303452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:25.303452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:25.303452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:25.303452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:25.303452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:25.303452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:25.303452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:25.303452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:25.303452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:25.303452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:25.303452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:25.303452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:25.303452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:25.304392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:25.304392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:25.304392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:25.304392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:25.304392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:25.304392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:25.304392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:25.304392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:25.304392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:25.304392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:25.304392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:25.304392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:25.304392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:25.304392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:25.304392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:25.304392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:25.304392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:25.305288 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:25.305288 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:25.305288 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:25.305288 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:25.305288 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:25.305288 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:25.305288 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:25.305288 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:25.310915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 374/739] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:25.313714 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b988232-9fc7-4721-8bc3-4de2d6e917d3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:25.413400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3bf1e6-4f3a-4a25-8df7-361a5070f579 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:25.459104 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:25.479286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fdf98-31f3-4dee-9eb0-d0fcc5d6fc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:25.480295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fdf98-31f3-4dee-9eb0-d0fcc5d6fc58 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:25.480785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fdf98-31f3-4dee-9eb0-d0fcc5d6fc58 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:25.481252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fdf98-31f3-4dee-9eb0-d0fcc5d6fc58 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:25.482124 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22fdf98-31f3-4dee-9eb0-d0fcc5d6fc58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:25.482662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22fdf98-31f3-4dee-9eb0-d0fcc5d6fc58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:25.484220 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22fdf98-31f3-4dee-9eb0-d0fcc5d6fc58 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:25.487413 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b988232-9fc7-4721-8bc3-4de2d6e917d3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:25.498281 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 366/740] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:55:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:25.500803 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fdf98-31f3-4dee-9eb0-d0fcc5d6fc58 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:25.524574 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d05896-0900-44c8-aa92-112f223fa04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:25.525109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d05896-0900-44c8-aa92-112f223fa04d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:25.525389 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d05896-0900-44c8-aa92-112f223fa04d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:25.525660 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d05896-0900-44c8-aa92-112f223fa04d None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:25.526930 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d05896-0900-44c8-aa92-112f223fa04d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:25.527234 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d05896-0900-44c8-aa92-112f223fa04d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:25.528670 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d05896-0900-44c8-aa92-112f223fa04d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:25.541700 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 375/741] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:25.547617 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d05896-0900-44c8-aa92-112f223fa04d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:25.554441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:25.657224 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d05896-0900-44c8-aa92-112f223fa04d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:25.673907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fdf98-31f3-4dee-9eb0-d0fcc5d6fc58 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:25.693792 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030940c1-628c-4dbf-b4c3-624f8e9fa7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:25.694599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030940c1-628c-4dbf-b4c3-624f8e9fa7bb None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:25.694887 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030940c1-628c-4dbf-b4c3-624f8e9fa7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:25.695159 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030940c1-628c-4dbf-b4c3-624f8e9fa7bb None admin] PATH_INFO: `/v3/projects/d11f33af53374dffa61132642a7770f4/users/615cc36e21cc493b823e6cee695ac308/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:25.697200 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030940c1-628c-4dbf-b4c3-624f8e9fa7bb None admin] RBAC: Authorizing `identity:create_grant(project_id=d11f33af53374dffa61132642a7770f4, user_id=615cc36e21cc493b823e6cee695ac308, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:25.699872 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030940c1-628c-4dbf-b4c3-624f8e9fa7bb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:25.719666 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217d5b2-4a31-4472-bf9d-35d500dbbe71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:25.720291 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217d5b2-4a31-4472-bf9d-35d500dbbe71 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:25.720560 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217d5b2-4a31-4472-bf9d-35d500dbbe71 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:25.720826 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217d5b2-4a31-4472-bf9d-35d500dbbe71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:25.735708 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217d5b2-4a31-4472-bf9d-35d500dbbe71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:25.737216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217d5b2-4a31-4472-bf9d-35d500dbbe71 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:25.761333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-a217d5b2-4a31-4472-bf9d-35d500dbbe71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:25.761333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:25.761333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:25.761333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:25.761333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:25.761333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:25.761333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:25.761333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:25.761333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:25.761333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:25.761333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:25.761333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:25.761333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:25.761333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:25.761333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:25.761333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:25.762748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:25.762748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:25.762748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:25.762748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:25.762748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:25.762748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:25.762748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:25.762748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:25.762748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:25.762748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:25.762748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:25.762748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:25.762748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:25.762748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:25.762748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:25.762748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:25.762748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:25.763651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:25.763651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:25.763651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:25.763651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 367/742] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:25.763651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217d5b2-4a31-4472-bf9d-35d500dbbe71 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:25.784494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:25.793715 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 376/743] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 10 16:55:25 2022] PUT /identity/v3/projects/d11f33af53374dffa61132642a7770f4/users/615cc36e21cc493b823e6cee695ac308/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:25.844723 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf01ad-99a7-4ad3-b7ea-d6f0c11d3b36 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:25.845232 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf01ad-99a7-4ad3-b7ea-d6f0c11d3b36 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:25.845392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf01ad-99a7-4ad3-b7ea-d6f0c11d3b36 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:25.851116 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 377/744] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:25.852884 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79097aba-b8ca-4221-840d-fbbe295cd5fc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:25.853225 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79097aba-b8ca-4221-840d-fbbe295cd5fc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:25.853497 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79097aba-b8ca-4221-840d-fbbe295cd5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:25.875432 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-79097aba-b8ca-4221-840d-fbbe295cd5fc None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:25.876662 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-79097aba-b8ca-4221-840d-fbbe295cd5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:25.958097 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:25.972372 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217d5b2-4a31-4472-bf9d-35d500dbbe71 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:25.997087 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 378/745] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:55:26.020189 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a241cc-0088-49e1-b7ec-ae4a8fd6e0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:26.020805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a241cc-0088-49e1-b7ec-ae4a8fd6e0aa None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:26.021080 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a241cc-0088-49e1-b7ec-ae4a8fd6e0aa None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:26.021349 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a241cc-0088-49e1-b7ec-ae4a8fd6e0aa None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:26.022010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a241cc-0088-49e1-b7ec-ae4a8fd6e0aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:26.022307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a241cc-0088-49e1-b7ec-ae4a8fd6e0aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:26.023913 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a241cc-0088-49e1-b7ec-ae4a8fd6e0aa None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:26.030821 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79097aba-b8ca-4221-840d-fbbe295cd5fc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:26.039015 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 368/746] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:26.043617 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a241cc-0088-49e1-b7ec-ae4a8fd6e0aa None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:26.164723 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a241cc-0088-49e1-b7ec-ae4a8fd6e0aa None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:26.234610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3765f-04b1-48bc-afe1-b75f604d49b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:26.235300 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3765f-04b1-48bc-afe1-b75f604d49b6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:26.235623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3765f-04b1-48bc-afe1-b75f604d49b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:26.235906 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3765f-04b1-48bc-afe1-b75f604d49b6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:26.247473 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee3765f-04b1-48bc-afe1-b75f604d49b6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:26.248920 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee3765f-04b1-48bc-afe1-b75f604d49b6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:26.252905 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79097aba-b8ca-4221-840d-fbbe295cd5fc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:26.272770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-0ee3765f-04b1-48bc-afe1-b75f604d49b6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 16:55:26.272770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:26.272770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:26.272770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:26.272770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:26.272770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:26.272770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:26.272770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:26.272770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:26.272770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:26.272770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:26.272770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:26.272770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:26.272770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:26.272770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:26.272770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:26.273756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:26.273756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:26.273756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:26.273756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:26.273756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:26.273756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:26.273756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:26.273756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:26.273756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:26.273756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:26.273756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:26.273756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:26.273756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:26.273756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:26.273756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:26.273756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:26.273756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:26.273756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:26.275092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:26.275092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:26.275092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:26.275092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:26.275092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:26.275092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:26.275092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:26.275092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:26.275092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 16:55:26.275092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:26.280645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 369/747] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:55:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:26.285270 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3765f-04b1-48bc-afe1-b75f604d49b6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:26.316976 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:26.322937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:26.331421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea50cd0-cef3-4b5b-babb-34a8f5b00b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:26.332201 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea50cd0-cef3-4b5b-babb-34a8f5b00b0c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:26.332480 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea50cd0-cef3-4b5b-babb-34a8f5b00b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:26.332750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea50cd0-cef3-4b5b-babb-34a8f5b00b0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:26.357784 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea50cd0-cef3-4b5b-babb-34a8f5b00b0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:26.364280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0ea50cd0-cef3-4b5b-babb-34a8f5b00b0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:26.364280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:26.364280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:26.364280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:26.364280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:26.364280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:26.364280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:26.364280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:26.364280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:26.364280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:26.364280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:26.364280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:26.364280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:26.364280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:26.364280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:26.364280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:26.365242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:26.365242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:26.365242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:26.365242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:26.365242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:26.365242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:26.365242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:26.365242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:26.365242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:26.365242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:26.365242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:26.365242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:26.365242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:26.365242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:26.365242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:26.365242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:26.366088 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:26.366088 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:26.366088 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:26.366088 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:26.366088 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:26.366088 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:26.366088 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:26.366088 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:26.366088 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:26.366088 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:26.366088 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:26.366088 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:26.370599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 379/748] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 10 16:55:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:55:26.373319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea50cd0-cef3-4b5b-babb-34a8f5b00b0c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:26.503250 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3765f-04b1-48bc-afe1-b75f604d49b6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:26.530550 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea50cd0-cef3-4b5b-babb-34a8f5b00b0c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:26.577762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4de56e-5718-434c-9925-b05ddb1e14e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:26.578296 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4de56e-5718-434c-9925-b05ddb1e14e8 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:26.578601 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4de56e-5718-434c-9925-b05ddb1e14e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:26.578872 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4de56e-5718-434c-9925-b05ddb1e14e8 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:26.579608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4de56e-5718-434c-9925-b05ddb1e14e8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:26.579910 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4de56e-5718-434c-9925-b05ddb1e14e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:26.581337 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4de56e-5718-434c-9925-b05ddb1e14e8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:26.583556 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee40b9-2d42-49c5-a55e-84ece7de2402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:26.584162 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee40b9-2d42-49c5-a55e-84ece7de2402 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:26.584683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee40b9-2d42-49c5-a55e-84ece7de2402 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:26.585140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee40b9-2d42-49c5-a55e-84ece7de2402 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:26.585886 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee40b9-2d42-49c5-a55e-84ece7de2402 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:26.586196 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee40b9-2d42-49c5-a55e-84ece7de2402 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:26.587581 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee40b9-2d42-49c5-a55e-84ece7de2402 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:26.605364 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 380/749] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 10 16:55:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:26.611353 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee40b9-2d42-49c5-a55e-84ece7de2402 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:26.624938 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 370/750] 10.222.0.45 () {56 vars in 1135 bytes} [Wed Aug 10 16:55:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:26.630880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4de56e-5718-434c-9925-b05ddb1e14e8 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:26.712968 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:26.760067 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:26.796685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4de56e-5718-434c-9925-b05ddb1e14e8 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:26.808691 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee40b9-2d42-49c5-a55e-84ece7de2402 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:26.838226 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c77ec-3dad-4871-9be9-f9f5610aa0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:26.839223 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c77ec-3dad-4871-9be9-f9f5610aa0db None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:26.839859 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c77ec-3dad-4871-9be9-f9f5610aa0db None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:26.841476 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c77ec-3dad-4871-9be9-f9f5610aa0db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:26.877042 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487660d5-7f6d-4ce3-8896-8c606fd5de4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:26.878839 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487660d5-7f6d-4ce3-8896-8c606fd5de4c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:26.879368 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487660d5-7f6d-4ce3-8896-8c606fd5de4c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:26.882742 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487660d5-7f6d-4ce3-8896-8c606fd5de4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:26.902749 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487660d5-7f6d-4ce3-8896-8c606fd5de4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:26.904217 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487660d5-7f6d-4ce3-8896-8c606fd5de4c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:26.920291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-487660d5-7f6d-4ce3-8896-8c606fd5de4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:26.920291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:26.920291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:26.920291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:26.920291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:26.920291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:26.920291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:26.920291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:26.920291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:26.920291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:26.920291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:26.920291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:26.920291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:26.920291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:26.920291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:26.920291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:26.921289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:26.921289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:26.921289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:26.921289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:26.921289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:26.921289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:26.921289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:26.921289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:26.921289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:26.921289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:26.921289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:26.921289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:26.921289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:26.921289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:26.921289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:26.921289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:26.921289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:26.922190 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:26.922190 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:26.922190 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:26.922190 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 381/751] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:26.927178 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487660d5-7f6d-4ce3-8896-8c606fd5de4c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:26.997694 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7c77ec-3dad-4871-9be9-f9f5610aa0db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:27.001120 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7c77ec-3dad-4871-9be9-f9f5610aa0db None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:27.095856 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487660d5-7f6d-4ce3-8896-8c606fd5de4c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:27.098241 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:27.111470 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:27.207187 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 371/752] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:55:26 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:55:27.207770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a653b5-88b7-44a8-9e41-212f0f46c4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:27.208644 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a653b5-88b7-44a8-9e41-212f0f46c4c5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:27.208822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f2cd7-5b3c-4686-bd3f-e5bf1ec507d0 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:27.209104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f2cd7-5b3c-4686-bd3f-e5bf1ec507d0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:27.209381 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f2cd7-5b3c-4686-bd3f-e5bf1ec507d0 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:27.209736 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a653b5-88b7-44a8-9e41-212f0f46c4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:27.210218 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a653b5-88b7-44a8-9e41-212f0f46c4c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:27.211126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a653b5-88b7-44a8-9e41-212f0f46c4c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:27.211623 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a653b5-88b7-44a8-9e41-212f0f46c4c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:27.213185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a653b5-88b7-44a8-9e41-212f0f46c4c5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:27.215725 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 372/753] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:27.216739 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f2cd7-5b3c-4686-bd3f-e5bf1ec507d0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:27.229802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 382/754] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:27.231783 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a1e40-d6fb-43ee-af04-ed6d305429ba None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:27.232288 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a1e40-d6fb-43ee-af04-ed6d305429ba None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:27.232758 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a1e40-d6fb-43ee-af04-ed6d305429ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:27.256235 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e05a1e40-d6fb-43ee-af04-ed6d305429ba None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:27.258448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e05a1e40-d6fb-43ee-af04-ed6d305429ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:27.346102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 383/755] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:55:27.347330 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a1e40-d6fb-43ee-af04-ed6d305429ba None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:27.358400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f2cd7-5b3c-4686-bd3f-e5bf1ec507d0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:27.401037 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64b308-7633-4d62-a25c-e56eaf76cb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:27.402676 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64b308-7633-4d62-a25c-e56eaf76cb5d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:27.403136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64b308-7633-4d62-a25c-e56eaf76cb5d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:27.403584 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64b308-7633-4d62-a25c-e56eaf76cb5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:27.414490 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb64b308-7633-4d62-a25c-e56eaf76cb5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:27.416294 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb64b308-7633-4d62-a25c-e56eaf76cb5d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:27.432642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-eb64b308-7633-4d62-a25c-e56eaf76cb5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:27.432642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:27.432642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:27.432642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:27.432642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:27.432642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:27.432642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:27.432642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:27.432642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:27.432642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:27.432642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:27.432642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:27.432642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:27.432642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:27.432642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:27.432642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:27.433792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:27.433792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:27.433792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:27.433792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:27.433792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:27.433792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:27.433792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:27.433792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:27.433792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:27.433792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:27.433792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:27.433792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:27.433792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:27.433792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:27.433792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:27.433792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:27.433792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:27.434730 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:27.434730 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:27.434730 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:27.436979 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 373/756] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:27.437992 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64b308-7633-4d62-a25c-e56eaf76cb5d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:27.461331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:27.467355 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:27.475857 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a1e40-d6fb-43ee-af04-ed6d305429ba None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:27.511274 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2946b88-67f3-4fff-98cc-94bad0347304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:27.511921 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2946b88-67f3-4fff-98cc-94bad0347304 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:27.512257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2946b88-67f3-4fff-98cc-94bad0347304 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:27.512504 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2946b88-67f3-4fff-98cc-94bad0347304 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:27.519722 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2946b88-67f3-4fff-98cc-94bad0347304 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:27.521186 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2946b88-67f3-4fff-98cc-94bad0347304 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:27.533508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c2946b88-67f3-4fff-98cc-94bad0347304 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 16:55:27.533508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:27.533508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:27.533508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:27.533508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:27.533508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:27.533508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:27.533508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:27.533508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:27.533508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:27.533508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:27.533508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:27.533508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:27.533508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:27.533508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:27.533508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:27.540597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:27.540597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:27.540597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:27.540597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:27.540597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:27.540597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:27.540597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:27.540597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:27.540597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:27.540597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:27.540597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:27.540597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:27.540597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:27.540597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:27.540597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:27.540597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:27.540597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:27.540597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:27.541553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:27.541553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:27.541553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:27.541553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:27.541553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:27.541553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:27.541553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:27.541553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:27.541553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 16:55:27.541553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:27.541553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 384/757] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 10 16:55:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:27.541553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48b244-8a61-4d18-99cc-28ef97d76c87 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:27.541553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48b244-8a61-4d18-99cc-28ef97d76c87 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:27.544627 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48b244-8a61-4d18-99cc-28ef97d76c87 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:27.544627 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 385/758] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:27.544627 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48b244-8a61-4d18-99cc-28ef97d76c87 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:27.574995 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64b308-7633-4d62-a25c-e56eaf76cb5d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:27.615875 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053abdd4-504f-4789-b8b1-14b0ef6d5355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:27.616563 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053abdd4-504f-4789-b8b1-14b0ef6d5355 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:27.616841 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053abdd4-504f-4789-b8b1-14b0ef6d5355 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:27.617104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053abdd4-504f-4789-b8b1-14b0ef6d5355 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:27.617678 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053abdd4-504f-4789-b8b1-14b0ef6d5355 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:27.619074 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053abdd4-504f-4789-b8b1-14b0ef6d5355 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:27.628797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-053abdd4-504f-4789-b8b1-14b0ef6d5355 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 16:55:27.628797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:27.628797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:55:27.628797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:55:27.628797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:55:27.628797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:55:27.628797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:55:27.628797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:27.628797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 16:55:27.628797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:27.628797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:55:27.628797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:27.628797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:27.628797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:27.628797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:27.628797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:27.628797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:27.629816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:27.629816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:27.629816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:27.629816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:27.629816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:27.629816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:27.629816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:55:27.629816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:55:27.629816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:55:27.629816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:55:27.629816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:27.629816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:27.629816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:27.629816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:27.629816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:27.629816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:27.630703 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:27.630703 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:27.630703 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:27.630703 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:27.630703 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:27.630703 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:27.630703 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:27.630703 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:27.630703 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:27.630703 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:27.630703 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:55:27.630703 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:27.630703 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 16:55:27.630703 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:27.634880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 374/759] 10.222.0.45 () {56 vars in 1129 bytes} [Wed Aug 10 16:55:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:27.636293 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053abdd4-504f-4789-b8b1-14b0ef6d5355 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:27.661534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48b244-8a61-4d18-99cc-28ef97d76c87 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:27.699404 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace6cd0-a5a9-48ad-bb11-2f948efec894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:27.700025 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace6cd0-a5a9-48ad-bb11-2f948efec894 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:27.700306 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace6cd0-a5a9-48ad-bb11-2f948efec894 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:27.700576 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace6cd0-a5a9-48ad-bb11-2f948efec894 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:27.701241 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace6cd0-a5a9-48ad-bb11-2f948efec894 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:27.701543 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace6cd0-a5a9-48ad-bb11-2f948efec894 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:27.702964 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace6cd0-a5a9-48ad-bb11-2f948efec894 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:27.715666 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 386/760] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:27.716888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1f0ca-d45f-401b-9f3f-5338639aaa53 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:27.717173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1f0ca-d45f-401b-9f3f-5338639aaa53 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:27.717442 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1f0ca-d45f-401b-9f3f-5338639aaa53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:27.726935 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:27.736067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-59a1f0ca-d45f-401b-9f3f-5338639aaa53 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:27.737265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-59a1f0ca-d45f-401b-9f3f-5338639aaa53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:27.749521 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053abdd4-504f-4789-b8b1-14b0ef6d5355 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:27.750281 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:27.792369 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce22aa-dd61-47c4-8eac-43853d56478b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:27.793041 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce22aa-dd61-47c4-8eac-43853d56478b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:27.793383 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce22aa-dd61-47c4-8eac-43853d56478b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:27.793935 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce22aa-dd61-47c4-8eac-43853d56478b None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:27.794785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce22aa-dd61-47c4-8eac-43853d56478b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:27.795157 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce22aa-dd61-47c4-8eac-43853d56478b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:27.796666 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce22aa-dd61-47c4-8eac-43853d56478b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:27.820270 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 375/761] 10.222.0.45 () {56 vars in 1149 bytes} [Wed Aug 10 16:55:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:27.820270 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce22aa-dd61-47c4-8eac-43853d56478b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:27.825758 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 387/762] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:55:27.826493 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1f0ca-d45f-401b-9f3f-5338639aaa53 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:27.939573 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce22aa-dd61-47c4-8eac-43853d56478b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:27.940209 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1f0ca-d45f-401b-9f3f-5338639aaa53 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:27.974970 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dae1ba-1c37-44aa-a7f3-fd1ad255c4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:27.975482 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2461f9c-a32a-4d76-b32b-cf55e7d14adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:27.975603 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dae1ba-1c37-44aa-a7f3-fd1ad255c4cf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:27.975852 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dae1ba-1c37-44aa-a7f3-fd1ad255c4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:27.976113 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dae1ba-1c37-44aa-a7f3-fd1ad255c4cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:27.976219 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2461f9c-a32a-4d76-b32b-cf55e7d14adb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:27.976598 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2461f9c-a32a-4d76-b32b-cf55e7d14adb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:27.976903 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2461f9c-a32a-4d76-b32b-cf55e7d14adb None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:27.977607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2461f9c-a32a-4d76-b32b-cf55e7d14adb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:27.977978 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2461f9c-a32a-4d76-b32b-cf55e7d14adb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:27.979584 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2461f9c-a32a-4d76-b32b-cf55e7d14adb None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:27.982813 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dae1ba-1c37-44aa-a7f3-fd1ad255c4cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:27.984528 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dae1ba-1c37-44aa-a7f3-fd1ad255c4cf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:27.984635 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 376/763] 10.222.0.45 () {56 vars in 1124 bytes} [Wed Aug 10 16:55:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:27.985216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2461f9c-a32a-4d76-b32b-cf55e7d14adb None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:27.989154 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:28.005319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-68dae1ba-1c37-44aa-a7f3-fd1ad255c4cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:28.005319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:28.005319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:28.005319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:28.005319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:28.005319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:28.005319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:28.005319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:28.005319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:28.005319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:28.005319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:28.005319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:28.005319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:28.005319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:28.005319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:28.006482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:28.006482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:28.006482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:28.006482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:28.006482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:28.006482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:28.006482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:28.006482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:28.006482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:28.006482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:28.006482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:28.006482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:28.006482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:28.006482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:28.006482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:28.006482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:28.006482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:28.007628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:28.007628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:28.007628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:28.007628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:28.007628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:28.007628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:28.007628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:28.007628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:28.014394 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 388/764] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:28.014394 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dae1ba-1c37-44aa-a7f3-fd1ad255c4cf None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:28.014394 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:28.099929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2461f9c-a32a-4d76-b32b-cf55e7d14adb None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:28.112509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dae1ba-1c37-44aa-a7f3-fd1ad255c4cf None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:28.136086 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46240ca0-e7c1-4792-be34-d7cc3ee87338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:28.136679 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46240ca0-e7c1-4792-be34-d7cc3ee87338 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:28.136953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46240ca0-e7c1-4792-be34-d7cc3ee87338 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:28.137218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46240ca0-e7c1-4792-be34-d7cc3ee87338 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:28.146448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e6164-d49e-49b1-b43c-bd438c5b13f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:28.147070 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e6164-d49e-49b1-b43c-bd438c5b13f5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:28.147349 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e6164-d49e-49b1-b43c-bd438c5b13f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:28.147626 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e6164-d49e-49b1-b43c-bd438c5b13f5 None admin] PATH_INFO: `/v3/services/network` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:28.148200 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e6164-d49e-49b1-b43c-bd438c5b13f5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:28.149829 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e6164-d49e-49b1-b43c-bd438c5b13f5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:28.158894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-551e6164-d49e-49b1-b43c-bd438c5b13f5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 16:55:28.158894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:28.158894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:55:28.158894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:55:28.158894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:55:28.158894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:55:28.158894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:55:28.158894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:28.158894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 16:55:28.158894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:28.158894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:55:28.158894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:28.158894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:28.158894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:28.158894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:28.158894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:28.158894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:28.160061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:28.160061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:28.160061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:28.160061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:28.160061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:28.160061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:28.160061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:55:28.160061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:55:28.160061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:55:28.160061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:55:28.160061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:28.160061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:28.160061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:28.160061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:28.160061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:28.160061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:28.161169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:28.161169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:28.161169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:28.161169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:28.161169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:28.161169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:28.161169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:28.161169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:28.161169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:28.161169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:28.161169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:55:28.161169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:28.161169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 16:55:28.161169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:28.161169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 389/765] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:28 2022] GET /identity/v3/services/network => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:28.161968 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e6164-d49e-49b1-b43c-bd438c5b13f5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:28.209332 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46240ca0-e7c1-4792-be34-d7cc3ee87338 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:28.210730 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46240ca0-e7c1-4792-be34-d7cc3ee87338 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:28.213371 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:28.230659 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:28.265041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e6164-d49e-49b1-b43c-bd438c5b13f5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:28.299449 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab2e8d-4ef4-4f95-b8ca-2bce23caf8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:28.300053 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab2e8d-4ef4-4f95-b8ca-2bce23caf8de None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:28.300334 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab2e8d-4ef4-4f95-b8ca-2bce23caf8de None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:28.300614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab2e8d-4ef4-4f95-b8ca-2bce23caf8de None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:28.301237 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab2e8d-4ef4-4f95-b8ca-2bce23caf8de None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:28.301536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab2e8d-4ef4-4f95-b8ca-2bce23caf8de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:28.302958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab2e8d-4ef4-4f95-b8ca-2bce23caf8de None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:28.309404 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 390/766] 10.222.0.45 () {56 vars in 1124 bytes} [Wed Aug 10 16:55:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:28.310623 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476fbb5-610d-4ae4-b5df-2d29b020f1a5 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:28.315423 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476fbb5-610d-4ae4-b5df-2d29b020f1a5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:28.315576 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 377/767] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:55:27 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:55:28.315688 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476fbb5-610d-4ae4-b5df-2d29b020f1a5 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:28.315781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46240ca0-e7c1-4792-be34-d7cc3ee87338 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:28.315862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 391/768] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:28.315862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476fbb5-610d-4ae4-b5df-2d29b020f1a5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:28.415529 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46240ca0-e7c1-4792-be34-d7cc3ee87338 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:28.418385 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:28.455168 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a6aa4-7b4d-4ae7-a0fb-f78fd748e3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:28.455782 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a6aa4-7b4d-4ae7-a0fb-f78fd748e3fa None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:28.456054 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a6aa4-7b4d-4ae7-a0fb-f78fd748e3fa None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:28.456321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a6aa4-7b4d-4ae7-a0fb-f78fd748e3fa None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:28.457020 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2a6aa4-7b4d-4ae7-a0fb-f78fd748e3fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:28.457320 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2a6aa4-7b4d-4ae7-a0fb-f78fd748e3fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:28.458814 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2a6aa4-7b4d-4ae7-a0fb-f78fd748e3fa None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:28.471712 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 378/769] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:28.472494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a6aa4-7b4d-4ae7-a0fb-f78fd748e3fa None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:28.539599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476fbb5-610d-4ae4-b5df-2d29b020f1a5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:28.567872 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:28.574172 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa4aed-94ec-4d81-8d93-fdb1ba63c74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:28.574799 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa4aed-94ec-4d81-8d93-fdb1ba63c74e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:28.575073 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa4aed-94ec-4d81-8d93-fdb1ba63c74e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:28.581668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a6aa4-7b4d-4ae7-a0fb-f78fd748e3fa None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:28.582133 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa4aed-94ec-4d81-8d93-fdb1ba63c74e None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:28.582133 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa4aed-94ec-4d81-8d93-fdb1ba63c74e None admin] RBAC: Adding query filter params (name=network) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:28.582133 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa4aed-94ec-4d81-8d93-fdb1ba63c74e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:28.582133 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa4aed-94ec-4d81-8d93-fdb1ba63c74e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:28.582429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 392/770] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 10 16:55:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:28.583143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa4aed-94ec-4d81-8d93-fdb1ba63c74e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:28.610901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7545268d-76b3-4d44-84a5-62d4ed60230b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:28.611505 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545268d-76b3-4d44-84a5-62d4ed60230b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:28.611788 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545268d-76b3-4d44-84a5-62d4ed60230b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:28.614818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545268d-76b3-4d44-84a5-62d4ed60230b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:28.614818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7545268d-76b3-4d44-84a5-62d4ed60230b None admin] RBAC: Adding query filter params (interface=public, service_id=ec47d0f1caaa4bf9a97cd50dab8fefa8, region_id=RegionOne) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:28.614818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7545268d-76b3-4d44-84a5-62d4ed60230b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:28.614818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7545268d-76b3-4d44-84a5-62d4ed60230b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:28.622050 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 379/771] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Aug 10 16:55:28 2022] GET /identity/v3/endpoints?service_id=ec47d0f1caaa4bf9a97cd50dab8fefa8&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:28.623362 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c4abb-09d7-4eb6-b303-45daea59944c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:28.623646 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c4abb-09d7-4eb6-b303-45daea59944c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:28.623918 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c4abb-09d7-4eb6-b303-45daea59944c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:28.641884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e74c4abb-09d7-4eb6-b303-45daea59944c None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:28.643101 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e74c4abb-09d7-4eb6-b303-45daea59944c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:28.666976 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:28.690235 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa4aed-94ec-4d81-8d93-fdb1ba63c74e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:28.725182 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c463d2-046f-4123-9b3d-12d8a7dd76b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:28.725724 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c463d2-046f-4123-9b3d-12d8a7dd76b6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:28.725999 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c463d2-046f-4123-9b3d-12d8a7dd76b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:28.726268 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c463d2-046f-4123-9b3d-12d8a7dd76b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:28.726826 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 380/772] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:55:28.727664 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c4abb-09d7-4eb6-b303-45daea59944c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:28.741273 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c463d2-046f-4123-9b3d-12d8a7dd76b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:28.743217 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c463d2-046f-4123-9b3d-12d8a7dd76b6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:28.758956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-b2c463d2-046f-4123-9b3d-12d8a7dd76b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:28.758956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:28.758956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:28.758956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:28.758956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:28.758956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:28.758956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:28.758956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:28.758956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:28.758956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:28.758956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:28.758956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:28.758956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:28.758956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:28.758956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:28.758956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:28.759958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:28.759958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:28.759958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:28.759958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:28.759958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:28.759958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:28.759958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:28.759958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:28.759958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:28.759958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:28.759958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:28.759958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:28.759958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:28.759958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:28.759958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:28.759958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:28.759958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:28.761082 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:28.761082 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:28.761082 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:28.761082 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 393/773] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:28.761082 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c463d2-046f-4123-9b3d-12d8a7dd76b6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:28.807330 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:28.831006 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c4abb-09d7-4eb6-b303-45daea59944c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:28.860635 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c463d2-046f-4123-9b3d-12d8a7dd76b6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:28.866226 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c728368-7582-48c6-b822-347af8690dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:28.866854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c728368-7582-48c6-b822-347af8690dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:28.867139 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c728368-7582-48c6-b822-347af8690dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:28.867411 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c728368-7582-48c6-b822-347af8690dd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:28.868127 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c728368-7582-48c6-b822-347af8690dd5 None admin] RBAC: Adding query filter params (role.id=7554b353873942fab506de830188aba0, scope.project.id=d11f33af53374dffa61132642a7770f4, user.id=4986821ef9f84b99a835e387635aa9a8) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:28.868431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c728368-7582-48c6-b822-347af8690dd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:28.870012 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c728368-7582-48c6-b822-347af8690dd5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:28.875287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 381/774] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:55:28 2022] GET /identity/v3/role_assignments?user.id=4986821ef9f84b99a835e387635aa9a8&scope.project.id=d11f33af53374dffa61132642a7770f4&role.id=7554b353873942fab506de830188aba0 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:28.876058 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c728368-7582-48c6-b822-347af8690dd5 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:28.894142 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b006779-1527-4688-af23-a9868ab24094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:28.894803 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b006779-1527-4688-af23-a9868ab24094 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:28.895082 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b006779-1527-4688-af23-a9868ab24094 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:28.895348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b006779-1527-4688-af23-a9868ab24094 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:28.895993 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b006779-1527-4688-af23-a9868ab24094 None admin] RBAC: Adding query filter params (type=network) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:28.896321 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b006779-1527-4688-af23-a9868ab24094 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:28.897741 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b006779-1527-4688-af23-a9868ab24094 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:28.904378 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 394/775] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 10 16:55:28 2022] GET /identity/v3/services?type=network => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:28.905254 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b006779-1527-4688-af23-a9868ab24094 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:28.921629 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:29.011230 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c728368-7582-48c6-b822-347af8690dd5 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:29.021526 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b006779-1527-4688-af23-a9868ab24094 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:29.032856 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:29.058252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a8943f-3a75-455f-a875-299e877e940b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:29.058909 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8943f-3a75-455f-a875-299e877e940b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:29.059186 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8943f-3a75-455f-a875-299e877e940b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:29.059525 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8943f-3a75-455f-a875-299e877e940b None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:29.060069 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a8943f-3a75-455f-a875-299e877e940b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:29.060986 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00dd698-b44e-4e8d-8614-9f16a781ac7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:29.061437 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a8943f-3a75-455f-a875-299e877e940b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:29.061609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dd698-b44e-4e8d-8614-9f16a781ac7c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:29.061887 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dd698-b44e-4e8d-8614-9f16a781ac7c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:29.062156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dd698-b44e-4e8d-8614-9f16a781ac7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:29.066784 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 382/776] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 10 16:55:28 2022] GET /identity/v3/services => generated 772 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:29.067565 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a8943f-3a75-455f-a875-299e877e940b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:29.095772 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00dd698-b44e-4e8d-8614-9f16a781ac7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:29.102244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d00dd698-b44e-4e8d-8614-9f16a781ac7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:55:29.102244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:29.102244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:29.102244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:29.102244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:29.102244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:29.102244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:29.102244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:29.102244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:29.102244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:29.102244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:29.102244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:29.102244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:29.102244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:29.102244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:29.102244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:29.103656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:29.103656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:29.103656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:29.103656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:29.103656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:29.103656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:29.103656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:29.103656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:29.103656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:29.103656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:29.103656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:29.103656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:29.103656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:29.103656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:29.103656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:29.103656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:29.105868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:29.105868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:29.105868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:29.105868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:29.105868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:29.105868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:29.105868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:29.105868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:29.105868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:29.105868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:29.105868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:55:29.105868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:29.110180 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 395/777] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:55:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:29.111224 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00dd698-b44e-4e8d-8614-9f16a781ac7c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:29.188913 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:29.207335 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a8943f-3a75-455f-a875-299e877e940b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:29.248674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47573b-f7d4-497a-8631-f3b404800bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:29.249275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47573b-f7d4-497a-8631-f3b404800bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:29.249550 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47573b-f7d4-497a-8631-f3b404800bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:29.249875 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47573b-f7d4-497a-8631-f3b404800bb9 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:29.253893 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47573b-f7d4-497a-8631-f3b404800bb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:29.253893 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47573b-f7d4-497a-8631-f3b404800bb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:29.253893 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47573b-f7d4-497a-8631-f3b404800bb9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:29.256448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00dd698-b44e-4e8d-8614-9f16a781ac7c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:29.267147 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 383/778] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:29.268002 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47573b-f7d4-497a-8631-f3b404800bb9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:29.303586 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c6d91-fa3f-4398-9423-0503acfae2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:29.304448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c6d91-fa3f-4398-9423-0503acfae2a3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:29.304723 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c6d91-fa3f-4398-9423-0503acfae2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:29.304993 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c6d91-fa3f-4398-9423-0503acfae2a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:29.305695 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7c6d91-fa3f-4398-9423-0503acfae2a3 None admin] RBAC: Adding query filter params (interface=public, service_id=53d9791454854c6f8a9152ae029f8356, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:29.305996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7c6d91-fa3f-4398-9423-0503acfae2a3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:29.307432 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7c6d91-fa3f-4398-9423-0503acfae2a3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:29.317232 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 396/779] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Aug 10 16:55:29 2022] GET /identity/v3/endpoints?service_id=53d9791454854c6f8a9152ae029f8356&interface=public®ion_id=RegionOne => generated 194 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:29.318274 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c6d91-fa3f-4398-9423-0503acfae2a3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:29.358029 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:29.420026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47573b-f7d4-497a-8631-f3b404800bb9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:29.438444 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c6d91-fa3f-4398-9423-0503acfae2a3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:29.462714 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a59e1-1366-4c5b-bab9-fe3efafccb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:29.463720 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a59e1-1366-4c5b-bab9-fe3efafccb82 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:29.464200 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a59e1-1366-4c5b-bab9-fe3efafccb82 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:29.464615 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a59e1-1366-4c5b-bab9-fe3efafccb82 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:29.465404 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a59e1-1366-4c5b-bab9-fe3efafccb82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:29.465850 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a59e1-1366-4c5b-bab9-fe3efafccb82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:29.467408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a59e1-1366-4c5b-bab9-fe3efafccb82 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:29.476991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 384/780] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:55:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:29.478135 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a59e1-1366-4c5b-bab9-fe3efafccb82 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:29.488068 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a94089-131f-463c-9ffc-74bbbc9f2906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:29.488716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a94089-131f-463c-9ffc-74bbbc9f2906 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:29.488998 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a94089-131f-463c-9ffc-74bbbc9f2906 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:29.489266 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a94089-131f-463c-9ffc-74bbbc9f2906 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:29.494759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:29.499915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a94089-131f-463c-9ffc-74bbbc9f2906 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:29.501605 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a94089-131f-463c-9ffc-74bbbc9f2906 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:29.514100 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-16a94089-131f-463c-9ffc-74bbbc9f2906 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:29.514100 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:29.514100 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:29.514100 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:29.514100 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:29.514100 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:29.514100 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:29.514100 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:29.514100 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:29.514100 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:29.514100 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:29.514100 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:29.514100 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:29.514100 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:29.514100 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:29.515309 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:29.515309 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:29.515309 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:29.515309 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:29.515309 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:29.515309 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:29.515309 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:29.515309 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:29.515309 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:29.515309 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:29.515309 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:29.515309 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:29.515309 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:29.515309 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:29.515309 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:29.515309 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:29.515309 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:29.516398 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:29.516398 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:29.516398 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:29.516398 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:29.516398 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:29.516398 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:29.516398 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:29.516398 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:29.523151 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 397/781] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:29.524573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a94089-131f-463c-9ffc-74bbbc9f2906 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:29.606894 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a59e1-1366-4c5b-bab9-fe3efafccb82 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:29.638624 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:29.646174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a94089-131f-463c-9ffc-74bbbc9f2906 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:29.647644 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c422d1-9bd0-4c1e-8c16-6659f9a4c25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:29.648305 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c422d1-9bd0-4c1e-8c16-6659f9a4c25a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:29.648582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c422d1-9bd0-4c1e-8c16-6659f9a4c25a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:29.648852 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c422d1-9bd0-4c1e-8c16-6659f9a4c25a None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:29.649726 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c422d1-9bd0-4c1e-8c16-6659f9a4c25a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:29.651944 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c422d1-9bd0-4c1e-8c16-6659f9a4c25a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:29.658112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 385/782] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 10 16:55:29 2022] GET /identity/v3/services => generated 772 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:29.659013 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c422d1-9bd0-4c1e-8c16-6659f9a4c25a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:29.712190 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa1f60-fecc-499e-8acf-08be723a344b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:29.712827 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa1f60-fecc-499e-8acf-08be723a344b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:29.713203 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa1f60-fecc-499e-8acf-08be723a344b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:29.713470 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa1f60-fecc-499e-8acf-08be723a344b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:29.738493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:29.739410 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aa1f60-fecc-499e-8acf-08be723a344b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:29.740886 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aa1f60-fecc-499e-8acf-08be723a344b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:29.769858 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-05aa1f60-fecc-499e-8acf-08be723a344b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:29.769858 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:29.769858 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:29.769858 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:29.769858 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:29.769858 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:29.769858 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:29.769858 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:29.769858 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:29.769858 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:29.769858 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:29.769858 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:29.769858 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:29.769858 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:29.769858 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:29.769858 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:29.770937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:29.770937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:29.770937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:29.770937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:29.770937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:29.770937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:29.770937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:29.770937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:29.770937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:29.770937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:29.770937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:29.770937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:29.770937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:29.770937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:29.770937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:29.770937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:29.770937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:29.772292 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:29.772292 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:29.772292 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:29.772292 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 398/783] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:29.773581 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c422d1-9bd0-4c1e-8c16-6659f9a4c25a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:29.780629 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa1f60-fecc-499e-8acf-08be723a344b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:29.817668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ebad8-a1a6-44dc-b7ba-a0ec0a561479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:29.818363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ebad8-a1a6-44dc-b7ba-a0ec0a561479 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:29.818659 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ebad8-a1a6-44dc-b7ba-a0ec0a561479 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:29.818932 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ebad8-a1a6-44dc-b7ba-a0ec0a561479 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:29.819636 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ebad8-a1a6-44dc-b7ba-a0ec0a561479 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:29.820009 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ebad8-a1a6-44dc-b7ba-a0ec0a561479 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:29.821459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ebad8-a1a6-44dc-b7ba-a0ec0a561479 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:29.836190 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 386/784] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:29.842418 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ebad8-a1a6-44dc-b7ba-a0ec0a561479 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:29.955047 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa1f60-fecc-499e-8acf-08be723a344b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:29.990074 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ebad8-a1a6-44dc-b7ba-a0ec0a561479 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:29.998095 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6cd84b-16ca-4440-950c-7c6149d17fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:29.998755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6cd84b-16ca-4440-950c-7c6149d17fec None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:29.999029 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6cd84b-16ca-4440-950c-7c6149d17fec None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:29.999294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6cd84b-16ca-4440-950c-7c6149d17fec None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:29.999952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6cd84b-16ca-4440-950c-7c6149d17fec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:30.000247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6cd84b-16ca-4440-950c-7c6149d17fec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:30.002438 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6cd84b-16ca-4440-950c-7c6149d17fec None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:30.020410 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 399/785] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:30.026904 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6cd84b-16ca-4440-950c-7c6149d17fec None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:30.033607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:30.037870 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:30.040539 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7bec67-83a3-4be7-a965-8e5e0149549b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:30.041387 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bec67-83a3-4be7-a965-8e5e0149549b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:30.042023 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bec67-83a3-4be7-a965-8e5e0149549b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:30.042588 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bec67-83a3-4be7-a965-8e5e0149549b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:30.043525 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7bec67-83a3-4be7-a965-8e5e0149549b None admin] RBAC: Adding query filter params (role.id=303e0fba379a41019ecc2e119d87fb1b, scope.project.id=d11f33af53374dffa61132642a7770f4, user.id=4f9e1767ec90435f80248a2930081265) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:30.044030 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7bec67-83a3-4be7-a965-8e5e0149549b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:30.045647 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7bec67-83a3-4be7-a965-8e5e0149549b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:30.051740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 387/786] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:55:29 2022] GET /identity/v3/role_assignments?user.id=4f9e1767ec90435f80248a2930081265&scope.project.id=d11f33af53374dffa61132642a7770f4&role.id=303e0fba379a41019ecc2e119d87fb1b => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:30.176981 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6cd84b-16ca-4440-950c-7c6149d17fec None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:30.216698 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dede7d-3a97-400a-acc4-61228c452226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:30.217342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dede7d-3a97-400a-acc4-61228c452226 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:30.217614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dede7d-3a97-400a-acc4-61228c452226 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:30.217880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dede7d-3a97-400a-acc4-61228c452226 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:30.226001 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dede7d-3a97-400a-acc4-61228c452226 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:30.227545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dede7d-3a97-400a-acc4-61228c452226 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:30.241373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-a7dede7d-3a97-400a-acc4-61228c452226 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:55:30.241373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:30.241373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:30.241373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:30.241373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:30.241373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:30.241373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:30.241373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:30.241373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:30.241373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:30.241373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:30.241373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:30.241373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:30.241373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:30.241373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:30.241373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:30.242595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:30.242595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:30.242595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:30.242595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:30.242595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:30.242595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:30.242595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:30.242595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:30.242595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:30.242595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:30.242595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:30.242595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:30.242595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:30.242595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:30.242595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:30.242595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:30.242595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:30.242595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:30.243763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:30.243763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:30.243763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:30.243763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:30.243763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:30.243763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:30.243763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:30.243763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:30.243763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:55:30.243763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:30.243763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 400/787] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 10 16:55:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:55:30.250227 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7bec67-83a3-4be7-a965-8e5e0149549b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:30.377929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7bec67-83a3-4be7-a965-8e5e0149549b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:30.423505 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d812cc-5a66-4a19-81b9-32f8336b4364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:30.424360 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d812cc-5a66-4a19-81b9-32f8336b4364 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:30.424807 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d812cc-5a66-4a19-81b9-32f8336b4364 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:30.425222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d812cc-5a66-4a19-81b9-32f8336b4364 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:30.426101 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d812cc-5a66-4a19-81b9-32f8336b4364 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:30.426576 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d812cc-5a66-4a19-81b9-32f8336b4364 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:30.428146 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d812cc-5a66-4a19-81b9-32f8336b4364 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:30.454092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 388/788] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 10 16:55:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:30.459498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dede7d-3a97-400a-acc4-61228c452226 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:30.541140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:30.578342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dede7d-3a97-400a-acc4-61228c452226 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:30.614319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307eb30-48df-46b0-a237-6db33f30064b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:30.617596 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307eb30-48df-46b0-a237-6db33f30064b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:30.617596 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307eb30-48df-46b0-a237-6db33f30064b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:30.617596 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307eb30-48df-46b0-a237-6db33f30064b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:30.688513 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307eb30-48df-46b0-a237-6db33f30064b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:30.689891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307eb30-48df-46b0-a237-6db33f30064b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:30.756949 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:30.793620 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 401/789] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:55:30 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:55:30.798919 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d812cc-5a66-4a19-81b9-32f8336b4364 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:30.874380 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:30.920546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d812cc-5a66-4a19-81b9-32f8336b4364 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:30.958403 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea49b60-d73f-49d5-bbd1-86e058e32bdc None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:30.958598 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea49b60-d73f-49d5-bbd1-86e058e32bdc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:30.958864 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea49b60-d73f-49d5-bbd1-86e058e32bdc None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:30.960334 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 402/790] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:55:30.965884 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a094dc3-5e0b-450a-b5d6-13c20cf06a79 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:30.966165 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a094dc3-5e0b-450a-b5d6-13c20cf06a79 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:30.966464 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a094dc3-5e0b-450a-b5d6-13c20cf06a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:30.976834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d3884-e7e9-413d-a365-be7042857e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:30.977477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d3884-e7e9-413d-a365-be7042857e10 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:30.977751 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d3884-e7e9-413d-a365-be7042857e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:30.978016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d3884-e7e9-413d-a365-be7042857e10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:30.989646 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d3884-e7e9-413d-a365-be7042857e10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:30.991266 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d3884-e7e9-413d-a365-be7042857e10 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:30.998583 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5a094dc3-5e0b-450a-b5d6-13c20cf06a79 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:31.000211 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5a094dc3-5e0b-450a-b5d6-13c20cf06a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:31.008082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d24d3884-e7e9-413d-a365-be7042857e10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:31.008082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:31.008082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:31.008082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:31.008082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:31.008082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:31.008082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:31.008082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:31.008082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:31.008082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:31.008082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:31.008082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:31.008082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:31.008082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:31.008082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:31.008082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:31.009026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:31.009026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:31.009026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:31.009026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:31.009026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:31.009026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:31.009026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:31.009026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:31.009026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:31.009026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:31.009026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:31.009026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:31.009026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:31.009026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:31.009026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:31.009026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:31.009026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:31.009917 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:31.009917 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:31.009917 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:31.010087 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 389/791] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:31.015560 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d3884-e7e9-413d-a365-be7042857e10 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:31.124633 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 403/792] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:55:31.148510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a094dc3-5e0b-450a-b5d6-13c20cf06a79 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:31.217361 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d3884-e7e9-413d-a365-be7042857e10 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:31.234692 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:31.274582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45acd0a0-b965-4d16-a7d7-939a9fbb7e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:31.275184 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45acd0a0-b965-4d16-a7d7-939a9fbb7e8f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:31.275458 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45acd0a0-b965-4d16-a7d7-939a9fbb7e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:31.275723 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45acd0a0-b965-4d16-a7d7-939a9fbb7e8f None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:31.277070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45acd0a0-b965-4d16-a7d7-939a9fbb7e8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:31.277384 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45acd0a0-b965-4d16-a7d7-939a9fbb7e8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:31.278943 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45acd0a0-b965-4d16-a7d7-939a9fbb7e8f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:31.292860 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 390/793] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:31.300382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45acd0a0-b965-4d16-a7d7-939a9fbb7e8f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:31.325676 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:31.334229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a094dc3-5e0b-450a-b5d6-13c20cf06a79 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:31.379418 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999ff64e-0c93-4c43-a432-375cc5c4192d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:31.381670 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ff64e-0c93-4c43-a432-375cc5c4192d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:31.382668 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ff64e-0c93-4c43-a432-375cc5c4192d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:31.383790 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ff64e-0c93-4c43-a432-375cc5c4192d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:31.433494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999ff64e-0c93-4c43-a432-375cc5c4192d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:31.451051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-999ff64e-0c93-4c43-a432-375cc5c4192d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:55:31.451051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:31.451051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:31.451051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:31.451051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:31.451051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:31.451051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:31.451051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:31.451051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:31.451051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:31.451051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:31.451051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:31.451051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:31.451051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:31.451051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:31.451051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:31.452056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:31.452056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:31.452056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:31.452056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:31.452056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:31.452056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:31.452056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:31.452056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:31.452056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:31.452056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:31.452056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:31.452056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:31.452056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:31.452056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:31.452056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:31.452056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:31.452897 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:31.452897 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:31.452897 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:31.452897 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:31.452897 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:31.452897 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:31.452897 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:31.452897 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:31.452897 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:31.452897 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:31.452897 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:55:31.452897 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:31.452897 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 404/794] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:55:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:55:31.471327 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999ff64e-0c93-4c43-a432-375cc5c4192d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:31.475810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45acd0a0-b965-4d16-a7d7-939a9fbb7e8f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:31.519370 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b0896-14d2-4cc3-a549-ddbbe6ea36cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:31.520886 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b0896-14d2-4cc3-a549-ddbbe6ea36cd None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:31.522220 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b0896-14d2-4cc3-a549-ddbbe6ea36cd None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:31.524243 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b0896-14d2-4cc3-a549-ddbbe6ea36cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:31.539216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b0896-14d2-4cc3-a549-ddbbe6ea36cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:31.540899 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b0896-14d2-4cc3-a549-ddbbe6ea36cd None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:31.576125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-396b0896-14d2-4cc3-a549-ddbbe6ea36cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:31.576125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:31.576125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:31.576125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:31.576125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:31.576125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:31.576125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:31.576125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:31.576125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:31.576125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:31.576125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:31.576125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:31.576125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:31.576125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:31.576125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:31.577118 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:31.577118 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:31.577118 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:31.577118 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:31.577118 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:31.577118 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:31.577118 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:31.577118 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:31.577118 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:31.577118 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:31.577118 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:31.577118 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:31.577118 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:31.577118 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:31.577118 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:31.577118 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:31.577118 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:31.578016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:31.578016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:31.578016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:31.578016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:31.578016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:31.578016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:31.578016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:31.578016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:31.578463 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 391/795] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:31.587288 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b0896-14d2-4cc3-a549-ddbbe6ea36cd None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:31.641792 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999ff64e-0c93-4c43-a432-375cc5c4192d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:31.655614 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:31.687227 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c711a8-7d55-43bf-b858-f4254f89e55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:31.688113 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c711a8-7d55-43bf-b858-f4254f89e55c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:31.688396 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c711a8-7d55-43bf-b858-f4254f89e55c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:31.688669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c711a8-7d55-43bf-b858-f4254f89e55c None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:31.689499 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c711a8-7d55-43bf-b858-f4254f89e55c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:31.690154 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c711a8-7d55-43bf-b858-f4254f89e55c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:31.691733 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c711a8-7d55-43bf-b858-f4254f89e55c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:31.705455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 405/796] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:55:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:31.706978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f31004-0568-4d1d-a7c1-fc8bea064629 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:31.707265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f31004-0568-4d1d-a7c1-fc8bea064629 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:31.707532 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f31004-0568-4d1d-a7c1-fc8bea064629 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:31.714247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 406/797] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:31.716022 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f31004-0568-4d1d-a7c1-fc8bea064629 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:31.750896 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:31.753945 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b0896-14d2-4cc3-a549-ddbbe6ea36cd None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:31.802622 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb84273c-f5bd-487b-be2f-a1eb66b862b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:31.803415 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84273c-f5bd-487b-be2f-a1eb66b862b2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:31.803986 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84273c-f5bd-487b-be2f-a1eb66b862b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:31.804304 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84273c-f5bd-487b-be2f-a1eb66b862b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:31.805898 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84273c-f5bd-487b-be2f-a1eb66b862b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:31.806233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84273c-f5bd-487b-be2f-a1eb66b862b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:31.809132 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84273c-f5bd-487b-be2f-a1eb66b862b2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:31.821828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 392/798] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:31.823238 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bdea7-fb51-4cdc-bb8f-7b0d8c35c184 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:31.824174 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bdea7-fb51-4cdc-bb8f-7b0d8c35c184 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:31.824446 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bdea7-fb51-4cdc-bb8f-7b0d8c35c184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:31.847883 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-876bdea7-fb51-4cdc-bb8f-7b0d8c35c184 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:31.848681 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f31004-0568-4d1d-a7c1-fc8bea064629 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:31.849065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-876bdea7-fb51-4cdc-bb8f-7b0d8c35c184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:31.886820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700d03a-8023-4a98-8115-00d357c3a54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:31.887494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700d03a-8023-4a98-8115-00d357c3a54c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:31.887835 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700d03a-8023-4a98-8115-00d357c3a54c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:31.888168 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700d03a-8023-4a98-8115-00d357c3a54c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:31.899430 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5700d03a-8023-4a98-8115-00d357c3a54c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:31.901008 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5700d03a-8023-4a98-8115-00d357c3a54c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:31.918978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5700d03a-8023-4a98-8115-00d357c3a54c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:31.918978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:31.918978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:31.918978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:31.918978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:31.918978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:31.918978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:31.918978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:31.918978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:31.918978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:31.918978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:31.918978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:31.918978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:31.918978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:31.918978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:31.918978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:31.921020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:31.921020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:31.921020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:31.921020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:31.921020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:31.921020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:31.921020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:31.921020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:31.921020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:31.921020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:31.921020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:31.921020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:31.921020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:31.921020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:31.921020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:31.921020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:31.921020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:31.923242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:31.923242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:31.923242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:31.923242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 407/799] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:31.923242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700d03a-8023-4a98-8115-00d357c3a54c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:31.930375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 393/800] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:55:31.935283 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bdea7-fb51-4cdc-bb8f-7b0d8c35c184 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:31.965717 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:31.996858 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:32.032103 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700d03a-8023-4a98-8115-00d357c3a54c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:32.066695 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bdea7-fb51-4cdc-bb8f-7b0d8c35c184 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:32.069331 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6159c7-a40e-4558-b560-7971a1a255c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:32.070156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6159c7-a40e-4558-b560-7971a1a255c9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:32.070636 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6159c7-a40e-4558-b560-7971a1a255c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:32.071055 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6159c7-a40e-4558-b560-7971a1a255c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:32.072012 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6159c7-a40e-4558-b560-7971a1a255c9 None admin] RBAC: Adding query filter params (role.id=7554b353873942fab506de830188aba0, scope.project.id=d11f33af53374dffa61132642a7770f4, user.id=615cc36e21cc493b823e6cee695ac308) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:32.072464 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6159c7-a40e-4558-b560-7971a1a255c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:32.074788 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6159c7-a40e-4558-b560-7971a1a255c9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:32.081880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 408/801] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:55:31 2022] GET /identity/v3/role_assignments?user.id=615cc36e21cc493b823e6cee695ac308&scope.project.id=d11f33af53374dffa61132642a7770f4&role.id=7554b353873942fab506de830188aba0 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:32.083334 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6159c7-a40e-4558-b560-7971a1a255c9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:32.108419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414a0cd-beb9-4e51-8043-119c400e492b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:32.109091 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414a0cd-beb9-4e51-8043-119c400e492b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:32.109443 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414a0cd-beb9-4e51-8043-119c400e492b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:32.109768 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414a0cd-beb9-4e51-8043-119c400e492b None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:32.110652 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414a0cd-beb9-4e51-8043-119c400e492b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:32.110921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414a0cd-beb9-4e51-8043-119c400e492b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:32.112441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414a0cd-beb9-4e51-8043-119c400e492b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:32.127907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 394/802] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:32.133377 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414a0cd-beb9-4e51-8043-119c400e492b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:32.202810 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6159c7-a40e-4558-b560-7971a1a255c9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:32.225801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:32.252702 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c060b8f-1ed6-40e8-93b9-c30fe2fce4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:32.253622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c060b8f-1ed6-40e8-93b9-c30fe2fce4b9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:32.254086 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c060b8f-1ed6-40e8-93b9-c30fe2fce4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:32.254581 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c060b8f-1ed6-40e8-93b9-c30fe2fce4b9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:32.255345 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c060b8f-1ed6-40e8-93b9-c30fe2fce4b9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:32.256948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c060b8f-1ed6-40e8-93b9-c30fe2fce4b9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:32.272490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-8c060b8f-1ed6-40e8-93b9-c30fe2fce4b9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 16:55:32.272490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:32.272490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:55:32.272490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:55:32.272490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:55:32.272490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:55:32.272490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:55:32.272490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:32.272490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 16:55:32.272490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:32.272490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:55:32.272490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:32.272490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:32.272490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:32.272490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:32.272490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:32.272490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:32.273748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:32.273748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:32.273748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:32.273748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:32.273748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:32.273748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:32.273748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:55:32.273748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:55:32.273748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:55:32.273748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:55:32.273748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:32.273748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:32.273748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:32.273748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:32.273748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:32.273748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:32.275161 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:32.275161 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:32.275161 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:32.275161 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:32.275161 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:32.275161 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:32.275161 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:32.275161 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:32.275161 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:32.275161 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:32.275161 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:55:32.275161 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:32.275161 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 16:55:32.275161 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:32.278176 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 409/803] 10.222.0.45 () {56 vars in 1129 bytes} [Wed Aug 10 16:55:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:32.278605 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:32.291132 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666802c-76ed-4787-8e32-f443145b7cc1 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:32.291132 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666802c-76ed-4787-8e32-f443145b7cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:32.291132 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666802c-76ed-4787-8e32-f443145b7cc1 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:32.291132 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 410/804] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:55:32.291132 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666802c-76ed-4787-8e32-f443145b7cc1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:32.292409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414a0cd-beb9-4e51-8043-119c400e492b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:32.345639 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c81535-f1b4-4c09-bfc9-90b5e2167b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:32.346649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c81535-f1b4-4c09-bfc9-90b5e2167b81 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:32.347134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c81535-f1b4-4c09-bfc9-90b5e2167b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:32.347630 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c81535-f1b4-4c09-bfc9-90b5e2167b81 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:32.357185 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c81535-f1b4-4c09-bfc9-90b5e2167b81 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:32.358978 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c81535-f1b4-4c09-bfc9-90b5e2167b81 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:32.374065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-b3c81535-f1b4-4c09-bfc9-90b5e2167b81 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 16:55:32.374065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:32.374065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:32.374065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:32.374065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:32.374065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:32.374065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:32.374065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:32.374065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:32.374065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:32.374065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:32.374065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:32.374065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:32.374065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:32.374065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:32.374065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:32.375104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:32.375104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:32.375104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:32.375104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:32.375104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:32.375104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:32.375104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:32.375104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:32.375104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:32.375104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:32.375104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:32.375104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:32.375104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:32.375104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:32.375104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:32.375104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:32.375104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:32.375104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:32.376047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:32.376047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:32.376047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:32.376047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:32.376047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:32.376047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:32.376047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:32.376047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:32.376047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 16:55:32.376047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:32.378592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 395/805] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:55:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:55:32.379941 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750139f-801d-4097-87f3-94e01685ad90 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:32.380426 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750139f-801d-4097-87f3-94e01685ad90 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:32.380882 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750139f-801d-4097-87f3-94e01685ad90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:32.406259 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f750139f-801d-4097-87f3-94e01685ad90 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:32.407754 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f750139f-801d-4097-87f3-94e01685ad90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:32.420141 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666802c-76ed-4787-8e32-f443145b7cc1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:32.460399 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73394df-adcd-43a6-8b14-332f77252a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:32.461326 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73394df-adcd-43a6-8b14-332f77252a20 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:32.461602 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73394df-adcd-43a6-8b14-332f77252a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:32.461871 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73394df-adcd-43a6-8b14-332f77252a20 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:32.462570 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73394df-adcd-43a6-8b14-332f77252a20 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:32.462867 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73394df-adcd-43a6-8b14-332f77252a20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:32.465392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73394df-adcd-43a6-8b14-332f77252a20 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:32.471043 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 411/806] 10.222.0.45 () {56 vars in 1124 bytes} [Wed Aug 10 16:55:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:32.472382 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73394df-adcd-43a6-8b14-332f77252a20 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:32.487634 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 396/807] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:55:32.488900 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f750139f-801d-4097-87f3-94e01685ad90 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:32.553847 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:32.574716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:32.595247 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f750139f-801d-4097-87f3-94e01685ad90 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:32.617802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73394df-adcd-43a6-8b14-332f77252a20 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:32.644551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6faa3b3-72cc-45ef-8a72-3e406547cd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:32.645398 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6faa3b3-72cc-45ef-8a72-3e406547cd56 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:32.645680 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6faa3b3-72cc-45ef-8a72-3e406547cd56 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:32.646001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6faa3b3-72cc-45ef-8a72-3e406547cd56 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:32.647130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6faa3b3-72cc-45ef-8a72-3e406547cd56 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:32.647434 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6faa3b3-72cc-45ef-8a72-3e406547cd56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:32.649315 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6faa3b3-72cc-45ef-8a72-3e406547cd56 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:32.657289 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 397/808] 10.222.0.45 () {56 vars in 1124 bytes} [Wed Aug 10 16:55:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:32.658131 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6faa3b3-72cc-45ef-8a72-3e406547cd56 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:32.660367 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b48423-a392-43f1-907f-7156c72b8ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:32.661227 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b48423-a392-43f1-907f-7156c72b8ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:32.661793 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b48423-a392-43f1-907f-7156c72b8ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:32.662288 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b48423-a392-43f1-907f-7156c72b8ad2 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:32.663368 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b48423-a392-43f1-907f-7156c72b8ad2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:32.663871 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b48423-a392-43f1-907f-7156c72b8ad2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:32.665520 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b48423-a392-43f1-907f-7156c72b8ad2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:32.690059 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 412/809] 10.222.0.45 () {56 vars in 1135 bytes} [Wed Aug 10 16:55:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:32.691715 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b48423-a392-43f1-907f-7156c72b8ad2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:32.783269 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6faa3b3-72cc-45ef-8a72-3e406547cd56 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:32.816528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:32.818655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b48423-a392-43f1-907f-7156c72b8ad2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:32.824510 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2aa40-00b0-45e3-a193-60a2e003baf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:32.825620 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2aa40-00b0-45e3-a193-60a2e003baf9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:32.826096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2aa40-00b0-45e3-a193-60a2e003baf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:32.826564 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2aa40-00b0-45e3-a193-60a2e003baf9 None admin] PATH_INFO: `/v3/services/network` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:32.833092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b2aa40-00b0-45e3-a193-60a2e003baf9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:32.833092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b2aa40-00b0-45e3-a193-60a2e003baf9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:32.838029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-81b2aa40-00b0-45e3-a193-60a2e003baf9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 16:55:32.838029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:32.838029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:55:32.838029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:55:32.838029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:55:32.838029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:55:32.838029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:55:32.838029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:32.838029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 16:55:32.838029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:32.838029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:55:32.838029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:32.838029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:32.838029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:32.838029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:32.838029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:32.838029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:32.839229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:32.839229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:32.839229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:32.839229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:32.839229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:32.839229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:32.839229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:55:32.839229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:55:32.839229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:55:32.839229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:55:32.839229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:32.839229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:32.839229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:32.839229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:32.839229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:32.839229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:32.840109 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:32.840109 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:32.840109 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:32.840109 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:32.840109 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:32.840109 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:32.840109 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:32.840109 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:32.840109 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:32.840109 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:32.840109 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:55:32.840109 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:32.840109 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 16:55:32.840109 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:32.841066 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 398/810] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:32 2022] GET /identity/v3/services/network => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:32.842460 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2aa40-00b0-45e3-a193-60a2e003baf9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:32.846589 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:32.866481 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236c2c2-75f8-4079-9e39-947413952f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:32.867412 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236c2c2-75f8-4079-9e39-947413952f83 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:32.868292 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236c2c2-75f8-4079-9e39-947413952f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:32.868740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236c2c2-75f8-4079-9e39-947413952f83 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:32.870334 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236c2c2-75f8-4079-9e39-947413952f83 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:32.872748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236c2c2-75f8-4079-9e39-947413952f83 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:32.946098 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2aa40-00b0-45e3-a193-60a2e003baf9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:32.981485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ba48a-5c50-496d-95d3-4fbb8598e2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:32.982115 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ba48a-5c50-496d-95d3-4fbb8598e2e6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:32.982491 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ba48a-5c50-496d-95d3-4fbb8598e2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:32.982767 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ba48a-5c50-496d-95d3-4fbb8598e2e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:32.993418 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ba48a-5c50-496d-95d3-4fbb8598e2e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:32.995534 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ba48a-5c50-496d-95d3-4fbb8598e2e6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:33.010803 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 413/811] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 16:55:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:33.011943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236c2c2-75f8-4079-9e39-947413952f83 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:33.012531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d12ba48a-5c50-496d-95d3-4fbb8598e2e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:33.012531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:33.012531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:33.012531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:33.012531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:33.012531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:33.012531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:33.012531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:33.012531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:33.012531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:33.012531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:33.012531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:33.012531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:33.012531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:33.012531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:33.012531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:33.013441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:33.013441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:33.013441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:33.013441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:33.013441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:33.013441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:33.013441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:33.013441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:33.013441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:33.013441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:33.013441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:33.013441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:33.013441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:33.013441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:33.013441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:33.013441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:33.013441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:33.014323 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:33.014323 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:33.014323 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:33.018674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 399/812] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:33.021049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f34bc9-9b1e-498f-bfdb-af417c2c5814 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:33.021611 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f34bc9-9b1e-498f-bfdb-af417c2c5814 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:33.022136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f34bc9-9b1e-498f-bfdb-af417c2c5814 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:33.023762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 400/813] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:33.025062 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f34bc9-9b1e-498f-bfdb-af417c2c5814 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:33.097075 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:33.167451 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f34bc9-9b1e-498f-bfdb-af417c2c5814 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:33.176355 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236c2c2-75f8-4079-9e39-947413952f83 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:33.221243 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8689a8-4884-4500-8d9c-92b88f232940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:33.221852 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8689a8-4884-4500-8d9c-92b88f232940 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:33.222125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8689a8-4884-4500-8d9c-92b88f232940 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:33.222415 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8689a8-4884-4500-8d9c-92b88f232940 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:33.223409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8689a8-4884-4500-8d9c-92b88f232940 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:33.223717 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8689a8-4884-4500-8d9c-92b88f232940 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:33.224421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f2d94-9d8f-46b5-9d20-e78324dd045e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:33.225045 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f2d94-9d8f-46b5-9d20-e78324dd045e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:33.225148 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8689a8-4884-4500-8d9c-92b88f232940 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:33.225362 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f2d94-9d8f-46b5-9d20-e78324dd045e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:33.225641 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f2d94-9d8f-46b5-9d20-e78324dd045e None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:33.226270 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f2d94-9d8f-46b5-9d20-e78324dd045e None admin] RBAC: Adding query filter params (name=network) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:33.226866 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f2d94-9d8f-46b5-9d20-e78324dd045e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:33.228776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f2d94-9d8f-46b5-9d20-e78324dd045e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:33.234428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 414/814] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 10 16:55:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:33.237004 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067016af-5881-4bed-ab28-389c9d4ab520 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:33.237285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067016af-5881-4bed-ab28-389c9d4ab520 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:33.237871 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067016af-5881-4bed-ab28-389c9d4ab520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:33.239570 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 401/815] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:33.241289 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8689a8-4884-4500-8d9c-92b88f232940 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:33.259613 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-067016af-5881-4bed-ab28-389c9d4ab520 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:33.261688 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-067016af-5881-4bed-ab28-389c9d4ab520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:33.302753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:33.350392 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8689a8-4884-4500-8d9c-92b88f232940 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:33.351900 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:33.361678 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 415/816] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:55:33.362807 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067016af-5881-4bed-ab28-389c9d4ab520 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:33.405656 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9059e5-85e9-417f-bce6-f5281eeaca54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:33.406730 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9059e5-85e9-417f-bce6-f5281eeaca54 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:33.407209 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9059e5-85e9-417f-bce6-f5281eeaca54 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:33.407677 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9059e5-85e9-417f-bce6-f5281eeaca54 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:33.408567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9059e5-85e9-417f-bce6-f5281eeaca54 None admin] RBAC: Adding query filter params (type=network) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:33.409989 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9059e5-85e9-417f-bce6-f5281eeaca54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:33.411725 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9059e5-85e9-417f-bce6-f5281eeaca54 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:33.418512 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 402/817] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 10 16:55:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:33.419637 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9059e5-85e9-417f-bce6-f5281eeaca54 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:33.488767 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067016af-5881-4bed-ab28-389c9d4ab520 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:33.527374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9059e5-85e9-417f-bce6-f5281eeaca54 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:33.531931 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21fd96-3658-4221-b5ed-14f63c9213a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:33.532569 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21fd96-3658-4221-b5ed-14f63c9213a5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:33.532846 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21fd96-3658-4221-b5ed-14f63c9213a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:33.533114 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21fd96-3658-4221-b5ed-14f63c9213a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:33.541252 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe21fd96-3658-4221-b5ed-14f63c9213a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:33.543596 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe21fd96-3658-4221-b5ed-14f63c9213a5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:33.558173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-fe21fd96-3658-4221-b5ed-14f63c9213a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:33.558173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:33.558173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:33.558173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:33.558173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:33.558173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:33.558173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:33.558173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:33.558173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:33.558173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:33.558173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:33.558173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:33.558173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:33.558173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:33.558173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:33.559149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:33.559149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:33.559149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:33.559149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:33.559149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:33.559149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:33.559149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:33.559149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:33.559149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:33.559149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:33.559149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:33.559149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:33.559149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:33.559149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:33.559149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:33.559149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:33.559149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:33.560019 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:33.560019 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:33.560019 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:33.560019 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:33.560019 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:33.560019 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:33.560019 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:33.560019 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:33.562419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 416/818] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:33.563431 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21fd96-3658-4221-b5ed-14f63c9213a5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:33.574991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84abc4-aac7-4051-8e01-774f398e60ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:33.575628 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84abc4-aac7-4051-8e01-774f398e60ce None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:33.575893 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84abc4-aac7-4051-8e01-774f398e60ce None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:33.576160 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84abc4-aac7-4051-8e01-774f398e60ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:33.586764 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed84abc4-aac7-4051-8e01-774f398e60ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:33.592497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-ed84abc4-aac7-4051-8e01-774f398e60ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:33.592497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:33.592497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:33.592497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:33.592497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:33.592497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:33.592497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:33.592497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:33.592497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:33.592497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:33.592497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:33.592497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:33.592497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:33.592497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:33.592497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:33.592497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:33.593480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:33.593480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:33.593480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:33.593480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:33.593480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:33.593480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:33.593480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:33.593480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:33.593480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:33.593480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:33.593480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:33.593480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:33.593480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:33.593480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:33.593480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:33.593480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:33.594328 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:33.594328 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:33.594328 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:33.594328 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:33.594328 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:33.594328 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:33.594328 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:33.594328 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:33.594328 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:33.594328 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:33.594328 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:33.594328 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:33.598447 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 403/819] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 10 16:55:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:33.598731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:33.599214 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84abc4-aac7-4051-8e01-774f398e60ce None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:33.635532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:33.705692 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21fd96-3658-4221-b5ed-14f63c9213a5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:33.711216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84abc4-aac7-4051-8e01-774f398e60ce None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:33.750005 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f4e16-31bb-4686-bb04-1e9e5ad4774e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:33.750675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f4e16-31bb-4686-bb04-1e9e5ad4774e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:33.750961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f4e16-31bb-4686-bb04-1e9e5ad4774e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:33.751214 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f4e16-31bb-4686-bb04-1e9e5ad4774e None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:33.751913 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f4e16-31bb-4686-bb04-1e9e5ad4774e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:33.752216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f4e16-31bb-4686-bb04-1e9e5ad4774e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:33.752834 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32161082-b965-4249-8d18-a0fa0f52ac67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:33.753670 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32161082-b965-4249-8d18-a0fa0f52ac67 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:33.753776 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f4e16-31bb-4686-bb04-1e9e5ad4774e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:33.754232 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32161082-b965-4249-8d18-a0fa0f52ac67 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:33.754719 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32161082-b965-4249-8d18-a0fa0f52ac67 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:33.755780 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32161082-b965-4249-8d18-a0fa0f52ac67 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:33.757782 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32161082-b965-4249-8d18-a0fa0f52ac67 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:33.772449 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 404/820] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:33.772449 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f4e16-31bb-4686-bb04-1e9e5ad4774e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:33.860595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 417/821] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 16:55:33 2022] POST /identity/v3/endpoints => generated 337 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:33.862071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32161082-b965-4249-8d18-a0fa0f52ac67 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:33.863172 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:33.903452 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f4e16-31bb-4686-bb04-1e9e5ad4774e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:33.976588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:34.008121 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32161082-b965-4249-8d18-a0fa0f52ac67 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:34.024847 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1bb2e-c239-41c4-8f9f-d48ca0d02a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:34.025907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1bb2e-c239-41c4-8f9f-d48ca0d02a0f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:34.027025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1bb2e-c239-41c4-8f9f-d48ca0d02a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:34.027715 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1bb2e-c239-41c4-8f9f-d48ca0d02a0f None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:34.028625 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1bb2e-c239-41c4-8f9f-d48ca0d02a0f None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:34.029137 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1bb2e-c239-41c4-8f9f-d48ca0d02a0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:34.030804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1bb2e-c239-41c4-8f9f-d48ca0d02a0f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:34.051239 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 405/822] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 10 16:55:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:34.057437 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1bb2e-c239-41c4-8f9f-d48ca0d02a0f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:34.090268 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7127189-3088-4fea-9985-83ea8b43f67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:34.091961 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7127189-3088-4fea-9985-83ea8b43f67e None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:34.092461 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7127189-3088-4fea-9985-83ea8b43f67e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:34.093627 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7127189-3088-4fea-9985-83ea8b43f67e None admin] PATH_INFO: `/v3/projects/d11f33af53374dffa61132642a7770f4/users/4986821ef9f84b99a835e387635aa9a8/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:34.096805 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7127189-3088-4fea-9985-83ea8b43f67e None admin] RBAC: Authorizing `identity:create_grant(project_id=d11f33af53374dffa61132642a7770f4, user_id=4986821ef9f84b99a835e387635aa9a8, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:34.099279 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7127189-3088-4fea-9985-83ea8b43f67e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:34.185797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1bb2e-c239-41c4-8f9f-d48ca0d02a0f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:34.212589 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 418/823] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 10 16:55:33 2022] PUT /identity/v3/projects/d11f33af53374dffa61132642a7770f4/users/4986821ef9f84b99a835e387635aa9a8/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:34.234768 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902ab95-9359-478f-8799-1ce6d0371350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:34.235829 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902ab95-9359-478f-8799-1ce6d0371350 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:34.236326 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902ab95-9359-478f-8799-1ce6d0371350 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:34.236807 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902ab95-9359-478f-8799-1ce6d0371350 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:34.250773 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902ab95-9359-478f-8799-1ce6d0371350 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:34.252615 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902ab95-9359-478f-8799-1ce6d0371350 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:34.256177 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:34.270883 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-3902ab95-9359-478f-8799-1ce6d0371350 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:34.270883 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:34.270883 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:34.270883 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:34.270883 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:34.270883 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:34.270883 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:34.270883 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:34.270883 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:34.270883 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:34.270883 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:34.270883 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:34.270883 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:34.270883 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:34.270883 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:34.270883 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:34.276394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:34.276394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:34.276394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:34.276394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:34.276394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:34.276394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:34.276394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:34.276394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:34.276394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:34.276394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:34.276394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:34.276394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:34.276394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:34.276394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:34.276394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:34.276394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:34.276394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:34.277281 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:34.277281 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:34.277281 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:34.277281 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 406/824] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:34.279366 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7127189-3088-4fea-9985-83ea8b43f67e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:34.374015 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d593d-e984-4ecf-8247-99993bfb770e None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:34.374391 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d593d-e984-4ecf-8247-99993bfb770e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:34.374608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d593d-e984-4ecf-8247-99993bfb770e None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:34.376256 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 407/825] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:55:34.381763 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41beb5-a93d-4880-9801-37a5ff9cac9d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:34.382058 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41beb5-a93d-4880-9801-37a5ff9cac9d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:34.382317 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41beb5-a93d-4880-9801-37a5ff9cac9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:34.404661 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7127189-3088-4fea-9985-83ea8b43f67e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:34.405095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0d41beb5-a93d-4880-9801-37a5ff9cac9d None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:34.406296 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0d41beb5-a93d-4880-9801-37a5ff9cac9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:34.444936 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac615043-6d99-41a8-9f4f-451fd252265c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:34.445469 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac615043-6d99-41a8-9f4f-451fd252265c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:34.445746 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac615043-6d99-41a8-9f4f-451fd252265c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:34.446011 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac615043-6d99-41a8-9f4f-451fd252265c None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:34.446687 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac615043-6d99-41a8-9f4f-451fd252265c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:34.446979 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac615043-6d99-41a8-9f4f-451fd252265c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:34.449094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac615043-6d99-41a8-9f4f-451fd252265c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:34.456285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:34.461364 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 419/826] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:34.468356 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac615043-6d99-41a8-9f4f-451fd252265c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:34.497172 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 408/827] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:34 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:55:34.519336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41beb5-a93d-4880-9801-37a5ff9cac9d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:34.607535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac615043-6d99-41a8-9f4f-451fd252265c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:34.641379 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41beb5-a93d-4880-9801-37a5ff9cac9d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:34.650904 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0ab73-959e-41a6-b1f3-c29f437f4452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:34.652064 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0ab73-959e-41a6-b1f3-c29f437f4452 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:34.652344 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0ab73-959e-41a6-b1f3-c29f437f4452 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:34.652615 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0ab73-959e-41a6-b1f3-c29f437f4452 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:34.663226 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb0ab73-959e-41a6-b1f3-c29f437f4452 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:34.665142 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb0ab73-959e-41a6-b1f3-c29f437f4452 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:34.676726 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:34.679753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ebb0ab73-959e-41a6-b1f3-c29f437f4452 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 16:55:34.679753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:34.679753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:34.679753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:34.679753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:34.679753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:34.679753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:34.679753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:34.679753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:34.679753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:34.679753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:34.679753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:34.679753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:34.679753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:34.679753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:34.679753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:34.680709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:34.680709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:34.680709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:34.680709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:34.680709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:34.680709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:34.680709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:34.680709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:34.680709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:34.680709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:34.680709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:34.680709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:34.680709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:34.680709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:34.680709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:34.680709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:34.680709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:34.680709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:34.681665 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:34.681665 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:34.681665 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:34.681665 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:34.681665 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:34.681665 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:34.681665 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:34.681665 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:34.681665 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 16:55:34.681665 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:34.681665 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 420/828] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 10 16:55:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:55:34.685317 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1a5da-3543-44bd-82fe-0c2255bdb555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:34.686074 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1a5da-3543-44bd-82fe-0c2255bdb555 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:34.686545 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1a5da-3543-44bd-82fe-0c2255bdb555 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:34.686748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1a5da-3543-44bd-82fe-0c2255bdb555 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:34.687593 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d1a5da-3543-44bd-82fe-0c2255bdb555 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:34.688023 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0ab73-959e-41a6-b1f3-c29f437f4452 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:34.689046 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d1a5da-3543-44bd-82fe-0c2255bdb555 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:34.699990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-89d1a5da-3543-44bd-82fe-0c2255bdb555 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 16:55:34.699990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:34.699990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:55:34.699990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:55:34.699990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:55:34.699990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:55:34.699990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:55:34.699990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:34.699990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 16:55:34.699990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:34.699990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:55:34.699990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:34.699990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:34.699990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:34.699990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:34.699990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:34.701016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:34.701016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:34.701016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:34.701016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:34.701016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:34.701016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:34.701016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:34.701016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:55:34.701016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:55:34.701016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:55:34.701016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:55:34.701016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:34.701016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:34.701016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:34.701016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:34.701016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:34.701016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:34.701907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:34.701907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:34.701907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:34.701907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:34.701907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:34.701907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:34.701907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:34.701907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:34.701907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:34.701907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:34.701907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:55:34.701907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:34.701907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 16:55:34.701907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:34.705897 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 409/829] 10.222.0.45 () {56 vars in 1132 bytes} [Wed Aug 10 16:55:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:55:34.707930 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1a5da-3543-44bd-82fe-0c2255bdb555 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:34.764405 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:34.821522 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0ab73-959e-41a6-b1f3-c29f437f4452 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:34.828388 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1a5da-3543-44bd-82fe-0c2255bdb555 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:34.865450 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667877b0-b58b-4570-a67e-61228965ef1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:34.866925 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667877b0-b58b-4570-a67e-61228965ef1e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:34.867488 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667877b0-b58b-4570-a67e-61228965ef1e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:34.868007 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667877b0-b58b-4570-a67e-61228965ef1e None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:34.868983 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4f286-21e3-4d4f-857b-e542cb4a26fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:34.869210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667877b0-b58b-4570-a67e-61228965ef1e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:34.869656 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667877b0-b58b-4570-a67e-61228965ef1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:34.870666 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4f286-21e3-4d4f-857b-e542cb4a26fb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:34.870927 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4f286-21e3-4d4f-857b-e542cb4a26fb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:34.871156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4f286-21e3-4d4f-857b-e542cb4a26fb None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:34.871980 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4f286-21e3-4d4f-857b-e542cb4a26fb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:34.872385 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4f286-21e3-4d4f-857b-e542cb4a26fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:34.872919 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667877b0-b58b-4570-a67e-61228965ef1e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:34.874000 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4f286-21e3-4d4f-857b-e542cb4a26fb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:34.880656 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 410/830] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 10 16:55:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:34.887059 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667877b0-b58b-4570-a67e-61228965ef1e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:34.899502 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 421/831] 10.222.0.45 () {56 vars in 1149 bytes} [Wed Aug 10 16:55:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:34.904878 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4f286-21e3-4d4f-857b-e542cb4a26fb None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:34.958411 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:35.013147 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667877b0-b58b-4570-a67e-61228965ef1e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:35.037207 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4f286-21e3-4d4f-857b-e542cb4a26fb None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:35.055986 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:35.065511 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b844f0-4328-42fa-a9ee-bd07163c9b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:35.066666 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b844f0-4328-42fa-a9ee-bd07163c9b9b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:35.067136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b844f0-4328-42fa-a9ee-bd07163c9b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:35.067720 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b844f0-4328-42fa-a9ee-bd07163c9b9b None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:35.068836 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b844f0-4328-42fa-a9ee-bd07163c9b9b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:35.069515 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b844f0-4328-42fa-a9ee-bd07163c9b9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:35.071404 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b844f0-4328-42fa-a9ee-bd07163c9b9b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:35.077387 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09bab85-20cc-4dab-86ce-9c086e08363c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:35.078155 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09bab85-20cc-4dab-86ce-9c086e08363c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:35.078590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09bab85-20cc-4dab-86ce-9c086e08363c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:35.086573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09bab85-20cc-4dab-86ce-9c086e08363c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:35.087066 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 411/832] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 10 16:55:34 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:35.099589 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09bab85-20cc-4dab-86ce-9c086e08363c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:35.101129 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09bab85-20cc-4dab-86ce-9c086e08363c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:35.118294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-e09bab85-20cc-4dab-86ce-9c086e08363c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:35.118294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:35.118294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:35.118294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:35.118294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:35.118294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:35.118294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:35.118294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:35.118294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:35.118294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:35.118294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:35.118294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:35.118294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:35.118294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:35.118294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:35.118294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:35.119415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:35.119415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:35.119415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:35.119415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:35.119415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:35.119415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:35.119415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:35.119415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:35.119415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:35.119415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:35.119415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:35.119415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:35.119415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:35.119415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:35.119415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:35.119415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:35.119415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:35.121348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:35.121348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:35.121348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:35.121348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 422/833] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:35.126031 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b844f0-4328-42fa-a9ee-bd07163c9b9b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:35.230459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2768fb-5329-4897-8aab-0ae4b7d79ecb None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:35.230459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2768fb-5329-4897-8aab-0ae4b7d79ecb None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:35.230459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2768fb-5329-4897-8aab-0ae4b7d79ecb None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:35.230459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 423/834] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:55:35.234767 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbb50c-26f6-468b-aed8-2c058509ff2f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:35.235060 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbb50c-26f6-468b-aed8-2c058509ff2f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:35.235312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbb50c-26f6-468b-aed8-2c058509ff2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:35.246366 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b844f0-4328-42fa-a9ee-bd07163c9b9b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:35.255165 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-13dbb50c-26f6-468b-aed8-2c058509ff2f None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:35.256551 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-13dbb50c-26f6-468b-aed8-2c058509ff2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:35.265933 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:35.284194 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274c175-a942-4c6a-ac3e-6f3b3a88d2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:35.285379 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274c175-a942-4c6a-ac3e-6f3b3a88d2ef None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:35.285379 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274c175-a942-4c6a-ac3e-6f3b3a88d2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:35.285597 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274c175-a942-4c6a-ac3e-6f3b3a88d2ef None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:35.286077 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2274c175-a942-4c6a-ac3e-6f3b3a88d2ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:35.287203 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2274c175-a942-4c6a-ac3e-6f3b3a88d2ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:35.288750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2274c175-a942-4c6a-ac3e-6f3b3a88d2ef None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:35.303827 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 412/835] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:35.311202 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274c175-a942-4c6a-ac3e-6f3b3a88d2ef None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:35.357838 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 424/836] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:55:35.379614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbb50c-26f6-468b-aed8-2c058509ff2f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:35.431879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274c175-a942-4c6a-ac3e-6f3b3a88d2ef None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:35.473157 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b2111-f657-43fb-bf00-33e3ecd941c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:35.473801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b2111-f657-43fb-bf00-33e3ecd941c7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:35.474077 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b2111-f657-43fb-bf00-33e3ecd941c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:35.474460 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b2111-f657-43fb-bf00-33e3ecd941c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:35.483122 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b2111-f657-43fb-bf00-33e3ecd941c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:35.484691 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b2111-f657-43fb-bf00-33e3ecd941c7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:35.493542 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:35.497743 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbb50c-26f6-468b-aed8-2c058509ff2f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:35.499590 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d73b2111-f657-43fb-bf00-33e3ecd941c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:35.499590 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:35.499590 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:35.499590 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:35.499590 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:35.499590 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:35.499590 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:35.499590 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:35.499590 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:35.499590 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:35.499590 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:35.499590 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:35.499590 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:35.499590 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:35.499590 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:35.500669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:35.500669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:35.500669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:35.500669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:35.500669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:35.500669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:35.500669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:35.500669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:35.500669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:35.500669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:35.500669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:35.500669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:35.500669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:35.500669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:35.500669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:35.500669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:35.500669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:35.501791 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:35.501791 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:35.501791 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:35.501791 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:35.501791 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:35.501791 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:35.501791 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:35.501791 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:35.503807 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 413/837] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:35.509381 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b2111-f657-43fb-bf00-33e3ecd941c7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:35.535297 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49974c2e-e50b-490d-b7f2-bbba6d1c286f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:35.535927 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49974c2e-e50b-490d-b7f2-bbba6d1c286f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:35.536203 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49974c2e-e50b-490d-b7f2-bbba6d1c286f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:35.536473 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49974c2e-e50b-490d-b7f2-bbba6d1c286f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:35.537162 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49974c2e-e50b-490d-b7f2-bbba6d1c286f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:35.538674 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49974c2e-e50b-490d-b7f2-bbba6d1c286f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:35.548655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-49974c2e-e50b-490d-b7f2-bbba6d1c286f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 16:55:35.548655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:35.548655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:55:35.548655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:55:35.548655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:55:35.548655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:55:35.548655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:55:35.548655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:35.548655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 16:55:35.548655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:35.548655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:55:35.548655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:35.548655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:35.548655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:35.548655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:35.548655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:35.548655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:35.549708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:35.549708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:35.549708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:35.549708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:35.549708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:35.549708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:35.549708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:55:35.549708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:55:35.549708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:55:35.549708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:55:35.549708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:35.549708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:35.549708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:35.549708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:35.549708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:35.549708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:35.550588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:35.550588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:35.550588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:35.550588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:35.550588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:35.550588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:35.550588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:35.550588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:35.550588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:35.550588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:35.550588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:55:35.550588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:35.550588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 16:55:35.550588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:35.550588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 425/838] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 10 16:55:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:55:35.557076 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:35.559208 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49974c2e-e50b-490d-b7f2-bbba6d1c286f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:35.629727 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b2111-f657-43fb-bf00-33e3ecd941c7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:35.664860 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb3027-a4d9-404c-b07c-3fb068dfd380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:35.665824 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb3027-a4d9-404c-b07c-3fb068dfd380 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:35.666335 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb3027-a4d9-404c-b07c-3fb068dfd380 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:35.667420 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb3027-a4d9-404c-b07c-3fb068dfd380 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:35.668307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb3027-a4d9-404c-b07c-3fb068dfd380 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:35.668893 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb3027-a4d9-404c-b07c-3fb068dfd380 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:35.670557 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb3027-a4d9-404c-b07c-3fb068dfd380 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:35.677249 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49974c2e-e50b-490d-b7f2-bbba6d1c286f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:35.688019 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 414/839] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:35.689705 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb3027-a4d9-404c-b07c-3fb068dfd380 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:35.714024 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d706be-4088-4f5d-9a70-f8ea87ae5ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:35.714677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d706be-4088-4f5d-9a70-f8ea87ae5ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:35.714955 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d706be-4088-4f5d-9a70-f8ea87ae5ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:35.715220 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d706be-4088-4f5d-9a70-f8ea87ae5ef3 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:35.716103 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d706be-4088-4f5d-9a70-f8ea87ae5ef3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:35.716295 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d706be-4088-4f5d-9a70-f8ea87ae5ef3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:35.717680 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d706be-4088-4f5d-9a70-f8ea87ae5ef3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:35.723093 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 426/840] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 10 16:55:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:35.728451 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d706be-4088-4f5d-9a70-f8ea87ae5ef3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:35.764924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:35.805598 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:35.810488 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb3027-a4d9-404c-b07c-3fb068dfd380 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:35.850514 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b4129-f098-4756-b10a-2ef9697caefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:35.851515 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b4129-f098-4756-b10a-2ef9697caefd None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:35.851974 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b4129-f098-4756-b10a-2ef9697caefd None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:35.852425 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b4129-f098-4756-b10a-2ef9697caefd None admin] PATH_INFO: `/v3/projects/d11f33af53374dffa61132642a7770f4/users/4f9e1767ec90435f80248a2930081265/roles/303e0fba379a41019ecc2e119d87fb1b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:35.866038 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d706be-4088-4f5d-9a70-f8ea87ae5ef3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:35.870462 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b4129-f098-4756-b10a-2ef9697caefd None admin] RBAC: Authorizing `identity:create_grant(project_id=d11f33af53374dffa61132642a7770f4, user_id=4f9e1767ec90435f80248a2930081265, role_id=303e0fba379a41019ecc2e119d87fb1b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:35.872607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b4129-f098-4756-b10a-2ef9697caefd None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:35.903509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d5aa8-8109-472a-a50c-5077def57d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:35.904224 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d5aa8-8109-472a-a50c-5077def57d9e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:35.904567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d5aa8-8109-472a-a50c-5077def57d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:35.904894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d5aa8-8109-472a-a50c-5077def57d9e None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:35.905590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d5aa8-8109-472a-a50c-5077def57d9e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:35.905955 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d5aa8-8109-472a-a50c-5077def57d9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:35.907447 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d5aa8-8109-472a-a50c-5077def57d9e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:35.912462 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 427/841] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 10 16:55:35 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:35.958613 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 415/842] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 10 16:55:35 2022] PUT /identity/v3/projects/d11f33af53374dffa61132642a7770f4/users/4f9e1767ec90435f80248a2930081265/roles/303e0fba379a41019ecc2e119d87fb1b => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:36.455026 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823f99d-6a75-4cca-bff8-3680d29d856b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:36.455405 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823f99d-6a75-4cca-bff8-3680d29d856b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:36.455605 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823f99d-6a75-4cca-bff8-3680d29d856b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:36.457205 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 428/843] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:36.462387 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5df16-1b5d-43a8-ac98-02249b578f9c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:36.462688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5df16-1b5d-43a8-ac98-02249b578f9c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:36.462962 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5df16-1b5d-43a8-ac98-02249b578f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:36.482652 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-95e5df16-1b5d-43a8-ac98-02249b578f9c None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:36.483846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-95e5df16-1b5d-43a8-ac98-02249b578f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:36.582059 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 416/844] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:55:36.626992 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823f99d-6a75-4cca-bff8-3680d29d856b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:36.793305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:36.812532 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823f99d-6a75-4cca-bff8-3680d29d856b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:36.871108 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276e988-20fb-4092-b476-585424632cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:36.872154 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276e988-20fb-4092-b476-585424632cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:36.872648 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276e988-20fb-4092-b476-585424632cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:36.873138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276e988-20fb-4092-b476-585424632cf7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:36.891296 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276e988-20fb-4092-b476-585424632cf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:36.901544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c276e988-20fb-4092-b476-585424632cf7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:55:36.901544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:36.901544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:36.901544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:36.901544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:36.901544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:36.901544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:36.901544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:36.901544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:36.901544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:36.901544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:36.901544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:36.901544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:36.901544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:36.901544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:36.901544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:36.902559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:36.902559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:36.902559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:36.902559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:36.902559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:36.902559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:36.902559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:36.902559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:36.902559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:36.902559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:36.902559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:36.902559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:36.902559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:36.902559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:36.902559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:36.902559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:36.903410 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:36.903410 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:36.903410 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:36.903410 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:36.903410 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:36.903410 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:36.903410 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:36.903410 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:36.903410 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:36.903410 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:36.903410 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:55:36.903410 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:36.904549 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 429/845] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:55:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:36.913112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5df16-1b5d-43a8-ac98-02249b578f9c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:36.996427 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:37.017938 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5df16-1b5d-43a8-ac98-02249b578f9c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:37.054007 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af6b95-3748-4232-9664-4f0d4183348f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:37.054560 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af6b95-3748-4232-9664-4f0d4183348f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:37.054837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af6b95-3748-4232-9664-4f0d4183348f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:37.055106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af6b95-3748-4232-9664-4f0d4183348f None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:37.055773 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af6b95-3748-4232-9664-4f0d4183348f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:37.056068 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af6b95-3748-4232-9664-4f0d4183348f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:37.058205 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af6b95-3748-4232-9664-4f0d4183348f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:37.067563 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 417/846] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:55:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:37.073178 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276e988-20fb-4092-b476-585424632cf7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:37.184930 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276e988-20fb-4092-b476-585424632cf7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:37.222171 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcea564-a755-4881-99b2-db04f889467c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:37.222829 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcea564-a755-4881-99b2-db04f889467c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:37.223106 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcea564-a755-4881-99b2-db04f889467c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:37.223372 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcea564-a755-4881-99b2-db04f889467c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:37.234558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcea564-a755-4881-99b2-db04f889467c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:37.236006 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcea564-a755-4881-99b2-db04f889467c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:37.248555 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36271e2f-8a0b-4b13-8724-d1da63bedb35 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:37.248837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36271e2f-8a0b-4b13-8724-d1da63bedb35 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:37.249104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36271e2f-8a0b-4b13-8724-d1da63bedb35 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:37.250596 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 418/847] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:55:37.254708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-6dcea564-a755-4881-99b2-db04f889467c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:37.254708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:37.254708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:37.254708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:37.254708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:37.254708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:37.254708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:37.254708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:37.254708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:37.254708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:37.254708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:37.254708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:37.254708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:37.254708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:37.254708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:37.254708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:37.255655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:37.255655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:37.255655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:37.255655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:37.255655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:37.255655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:37.255655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:37.255655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:37.255655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:37.255655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:37.255655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:37.255655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:37.255655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:37.255655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:37.255655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:37.255655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:37.255655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:37.256555 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:37.256555 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:37.256555 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:37.256555 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 430/848] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:37.258313 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b593d-a820-4cd8-b414-347039d23f38 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:37.258718 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b593d-a820-4cd8-b414-347039d23f38 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:37.259072 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b593d-a820-4cd8-b414-347039d23f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:37.261851 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcea564-a755-4881-99b2-db04f889467c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:37.266328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:37.278510 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-137b593d-a820-4cd8-b414-347039d23f38 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:37.279789 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-137b593d-a820-4cd8-b414-347039d23f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:37.357560 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 419/849] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:55:37.381085 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b593d-a820-4cd8-b414-347039d23f38 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:37.392329 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcea564-a755-4881-99b2-db04f889467c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:37.439792 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa2eb4-473f-4873-ae01-8d8d99c951b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:37.440399 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa2eb4-473f-4873-ae01-8d8d99c951b8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:37.440927 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa2eb4-473f-4873-ae01-8d8d99c951b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:37.441205 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa2eb4-473f-4873-ae01-8d8d99c951b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:37.442268 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa2eb4-473f-4873-ae01-8d8d99c951b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:37.442599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa2eb4-473f-4873-ae01-8d8d99c951b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:37.444514 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa2eb4-473f-4873-ae01-8d8d99c951b8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:37.457002 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:37.462509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 431/850] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:37.468576 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa2eb4-473f-4873-ae01-8d8d99c951b8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:37.492800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b593d-a820-4cd8-b414-347039d23f38 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:37.533181 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e203ff-65ef-4a4d-806d-db0afb1be535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:37.534009 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e203ff-65ef-4a4d-806d-db0afb1be535 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:37.534373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e203ff-65ef-4a4d-806d-db0afb1be535 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:37.534725 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e203ff-65ef-4a4d-806d-db0afb1be535 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:37.535501 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e203ff-65ef-4a4d-806d-db0afb1be535 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:37.536958 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e203ff-65ef-4a4d-806d-db0afb1be535 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:37.547940 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:37.586122 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa2eb4-473f-4873-ae01-8d8d99c951b8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:37.631160 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 420/851] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 10 16:55:37 2022] POST /identity/v3/services => generated 269 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:55:37.642515 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b6986-529b-427a-840e-d5596af0d168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:37.642515 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b6986-529b-427a-840e-d5596af0d168 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:37.642515 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b6986-529b-427a-840e-d5596af0d168 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:37.642515 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b6986-529b-427a-840e-d5596af0d168 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:37.650261 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b6986-529b-427a-840e-d5596af0d168 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:37.652716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b6986-529b-427a-840e-d5596af0d168 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:37.666844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0c2b6986-529b-427a-840e-d5596af0d168 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 16:55:37.666844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:37.666844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:37.666844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:37.666844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:37.666844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:37.666844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:37.666844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:37.666844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:37.666844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:37.666844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:37.666844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:37.666844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:37.666844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:37.666844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:37.666844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:37.668484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:37.668484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:37.668484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:37.668484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:37.668484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:37.668484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:37.668484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:37.668484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:37.668484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:37.668484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:37.668484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:37.668484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:37.668484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:37.668484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:37.668484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:37.668484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:37.668484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:37.668484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:37.669462 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:37.669462 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:37.669462 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:37.669462 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:37.669462 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:37.669462 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:37.669462 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:37.669462 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:37.669462 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 16:55:37.669462 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:37.669462 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 432/852] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:55:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:37.674419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e203ff-65ef-4a4d-806d-db0afb1be535 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:37.793721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e203ff-65ef-4a4d-806d-db0afb1be535 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:37.827657 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:37.834647 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b17c4-35c3-4899-a2fc-cc21b05df8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:37.835267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b17c4-35c3-4899-a2fc-cc21b05df8fc None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:37.835540 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b17c4-35c3-4899-a2fc-cc21b05df8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:37.835808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b17c4-35c3-4899-a2fc-cc21b05df8fc None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:37.836539 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b17c4-35c3-4899-a2fc-cc21b05df8fc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:37.836839 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b17c4-35c3-4899-a2fc-cc21b05df8fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:37.838276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b17c4-35c3-4899-a2fc-cc21b05df8fc None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:37.864369 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 421/853] 10.222.0.45 () {56 vars in 1135 bytes} [Wed Aug 10 16:55:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:37.871300 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b6986-529b-427a-840e-d5596af0d168 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:37.985192 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b6986-529b-427a-840e-d5596af0d168 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:38.004054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:38.022878 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfc9ad-855a-4bb8-a1df-ef2e7d9793f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:38.024128 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfc9ad-855a-4bb8-a1df-ef2e7d9793f0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:38.024558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfc9ad-855a-4bb8-a1df-ef2e7d9793f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:38.024979 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfc9ad-855a-4bb8-a1df-ef2e7d9793f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:38.043151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c9923-6d72-45ef-82df-0b479479dbd5 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:38.043681 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c9923-6d72-45ef-82df-0b479479dbd5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:38.043681 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c9923-6d72-45ef-82df-0b479479dbd5 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:38.046461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 422/854] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:55:38.050632 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc4f08-0332-4029-a668-64f0564cd144 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:38.050852 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc4f08-0332-4029-a668-64f0564cd144 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:38.051126 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc4f08-0332-4029-a668-64f0564cd144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:38.070166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-82fc4f08-0332-4029-a668-64f0564cd144 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:38.071407 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-82fc4f08-0332-4029-a668-64f0564cd144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:38.115734 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dfc9ad-855a-4bb8-a1df-ef2e7d9793f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:38.117498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dfc9ad-855a-4bb8-a1df-ef2e7d9793f0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:38.149265 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 423/855] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:38 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:55:38.150706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338fc9e-e6fd-4814-ace3-70a33cc289d0 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:38.151020 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338fc9e-e6fd-4814-ace3-70a33cc289d0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:38.151294 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338fc9e-e6fd-4814-ace3-70a33cc289d0 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:38.152947 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 424/856] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:38.158114 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab45766-1fff-4589-995b-034afd963300 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:38.158581 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab45766-1fff-4589-995b-034afd963300 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:38.158963 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab45766-1fff-4589-995b-034afd963300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:38.177282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5ab45766-1fff-4589-995b-034afd963300 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:38.178554 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5ab45766-1fff-4589-995b-034afd963300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:38.238398 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 433/857] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:55:37 2022] GET /identity/v3/auth/tokens => generated 1271 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:55:38.239666 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfc9ad-855a-4bb8-a1df-ef2e7d9793f0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:38.255285 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 425/858] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:38 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:55:38.256126 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab45766-1fff-4589-995b-034afd963300 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:38.270739 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:38.309765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:38.356973 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfc9ad-855a-4bb8-a1df-ef2e7d9793f0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:38.360601 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab45766-1fff-4589-995b-034afd963300 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:38.395067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dca08a-50d2-4d7a-aaa9-7d5040411903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:38.395619 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dca08a-50d2-4d7a-aaa9-7d5040411903 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:38.395900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dca08a-50d2-4d7a-aaa9-7d5040411903 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:38.396003 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6bf0d-4630-4219-bab2-10e1da126d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:38.396218 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dca08a-50d2-4d7a-aaa9-7d5040411903 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:38.396540 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6bf0d-4630-4219-bab2-10e1da126d35 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:38.396798 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dca08a-50d2-4d7a-aaa9-7d5040411903 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:38.396897 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6bf0d-4630-4219-bab2-10e1da126d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:38.397073 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6bf0d-4630-4219-bab2-10e1da126d35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:38.398245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dca08a-50d2-4d7a-aaa9-7d5040411903 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:38.408012 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6bf0d-4630-4219-bab2-10e1da126d35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:38.409474 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6bf0d-4630-4219-bab2-10e1da126d35 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:38.424706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-47e6bf0d-4630-4219-bab2-10e1da126d35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:38.424706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:38.424706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:38.424706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:38.424706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:38.424706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:38.424706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:38.424706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:38.424706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:38.424706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:38.424706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:38.424706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:38.424706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:38.424706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:38.424706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:38.424706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:38.425889 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:38.425889 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:38.425889 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:38.425889 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:38.425889 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:38.425889 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:38.425889 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:38.425889 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:38.425889 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:38.425889 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:38.425889 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:38.425889 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:38.425889 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:38.425889 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:38.425889 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:38.425889 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:38.425889 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:38.427016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:38.427016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:38.427016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:38.427016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 426/859] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:38.427016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6bf0d-4630-4219-bab2-10e1da126d35 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:38.526843 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:38.529811 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6bf0d-4630-4219-bab2-10e1da126d35 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:38.551000 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 434/860] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 10 16:55:38 2022] POST /identity/v3/services => generated 251 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:38.551827 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dca08a-50d2-4d7a-aaa9-7d5040411903 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:38.591647 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7f945-f197-4c6c-b777-63750187daea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:38.592662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7f945-f197-4c6c-b777-63750187daea None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:38.593175 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7f945-f197-4c6c-b777-63750187daea None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:38.593817 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7f945-f197-4c6c-b777-63750187daea None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:38.610634 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7f945-f197-4c6c-b777-63750187daea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:38.615688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-19a7f945-f197-4c6c-b777-63750187daea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:38.615688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:38.615688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:38.615688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:38.615688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:38.615688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:38.615688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:38.615688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:38.615688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:38.615688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:38.615688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:38.615688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:38.615688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:38.615688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:38.615688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:38.615688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:38.616706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:38.616706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:38.616706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:38.616706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:38.616706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:38.616706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:38.616706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:38.616706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:38.616706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:38.616706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:38.616706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:38.616706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:38.616706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:38.616706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:38.616706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:38.616706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:38.617608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:38.617608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:38.617608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:38.617608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:38.617608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:38.617608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:38.617608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:38.617608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:38.617608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:38.617608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:38.617608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:55:38.617608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:38.622946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 427/861] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 10 16:55:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:38.629086 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7f945-f197-4c6c-b777-63750187daea None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:38.679711 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:38.710199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dca08a-50d2-4d7a-aaa9-7d5040411903 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:38.752153 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9b4bf-15ee-49bf-ac40-f4e791f58d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:38.752787 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9b4bf-15ee-49bf-ac40-f4e791f58d62 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:38.753221 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9b4bf-15ee-49bf-ac40-f4e791f58d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:38.753609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9b4bf-15ee-49bf-ac40-f4e791f58d62 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:38.754274 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9b4bf-15ee-49bf-ac40-f4e791f58d62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:38.754610 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9b4bf-15ee-49bf-ac40-f4e791f58d62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:38.755313 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7f945-f197-4c6c-b777-63750187daea None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:38.756036 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9b4bf-15ee-49bf-ac40-f4e791f58d62 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:38.772913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 435/862] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:38.779588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9b4bf-15ee-49bf-ac40-f4e791f58d62 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:38.803120 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e7db2-5411-410e-bd51-191f7cc2238d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:38.803775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e7db2-5411-410e-bd51-191f7cc2238d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:38.804049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e7db2-5411-410e-bd51-191f7cc2238d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:38.804397 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e7db2-5411-410e-bd51-191f7cc2238d None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:38.805026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864e7db2-5411-410e-bd51-191f7cc2238d None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:38.805320 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864e7db2-5411-410e-bd51-191f7cc2238d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:38.806761 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864e7db2-5411-410e-bd51-191f7cc2238d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:38.816960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 428/863] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 10 16:55:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:38.827688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e7db2-5411-410e-bd51-191f7cc2238d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:38.854193 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:38.913611 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9b4bf-15ee-49bf-ac40-f4e791f58d62 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:38.959116 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172e0b5-100f-4a47-93ea-785a03dc2843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:38.959946 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172e0b5-100f-4a47-93ea-785a03dc2843 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:38.960223 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172e0b5-100f-4a47-93ea-785a03dc2843 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:38.960493 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172e0b5-100f-4a47-93ea-785a03dc2843 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:38.968713 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e7db2-5411-410e-bd51-191f7cc2238d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:38.969504 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2172e0b5-100f-4a47-93ea-785a03dc2843 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:38.972096 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2172e0b5-100f-4a47-93ea-785a03dc2843 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:38.974313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:38.989511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2172e0b5-100f-4a47-93ea-785a03dc2843 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:38.989511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:38.989511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:38.989511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:38.989511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:38.989511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:38.989511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:38.989511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:38.989511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:38.989511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:38.989511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:38.989511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:38.989511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:38.989511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:38.989511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:39.002584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:39.002584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:39.002584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:39.002584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:39.002584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:39.002584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:39.002584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:39.002584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:39.002584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:39.002584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:39.002584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:39.002584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:39.002584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:39.002584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:39.002584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:39.002584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:39.002584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:39.003509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:39.003509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:39.003509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:39.003509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:39.003509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:39.003509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:39.003509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:39.003509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:39.003509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 436/864] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:55:39.003509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172e0b5-100f-4a47-93ea-785a03dc2843 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:39.011866 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e60ee56-0c84-477d-918e-41b2037a55f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:39.012551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e60ee56-0c84-477d-918e-41b2037a55f4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:39.012832 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e60ee56-0c84-477d-918e-41b2037a55f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:39.013097 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e60ee56-0c84-477d-918e-41b2037a55f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:39.024426 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e60ee56-0c84-477d-918e-41b2037a55f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:39.025861 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e60ee56-0c84-477d-918e-41b2037a55f4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:39.041100 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-9e60ee56-0c84-477d-918e-41b2037a55f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:39.041100 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:39.041100 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:39.041100 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:39.041100 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:39.041100 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:39.041100 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:39.041100 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:39.041100 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:39.041100 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:39.041100 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:39.041100 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:39.041100 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:39.041100 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:39.041100 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:39.041100 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:39.043010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:39.043010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:39.043010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:39.043010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:39.043010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:39.043010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:39.043010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:39.043010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:39.043010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:39.043010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:39.043010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:39.043010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:39.043010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:39.043010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:39.043010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:39.043010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:39.043010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:39.044358 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:39.044358 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:39.044358 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:39.044358 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 429/865] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:39.050919 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e60ee56-0c84-477d-918e-41b2037a55f4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:39.133865 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172e0b5-100f-4a47-93ea-785a03dc2843 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:39.149079 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:39.170917 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921b836-1aa8-438e-912b-de797e1003e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:39.171567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921b836-1aa8-438e-912b-de797e1003e2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:39.171843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921b836-1aa8-438e-912b-de797e1003e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:39.172104 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921b836-1aa8-438e-912b-de797e1003e2 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:39.172830 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921b836-1aa8-438e-912b-de797e1003e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:39.173127 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921b836-1aa8-438e-912b-de797e1003e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:39.174586 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921b836-1aa8-438e-912b-de797e1003e2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:39.190403 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 437/866] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:39.195913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921b836-1aa8-438e-912b-de797e1003e2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:39.197869 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e60ee56-0c84-477d-918e-41b2037a55f4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:39.235019 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce549edb-6b6a-4f62-a6e0-0024ee29326a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:39.235850 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce549edb-6b6a-4f62-a6e0-0024ee29326a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:39.236337 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce549edb-6b6a-4f62-a6e0-0024ee29326a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:39.236804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce549edb-6b6a-4f62-a6e0-0024ee29326a None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:39.237680 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce549edb-6b6a-4f62-a6e0-0024ee29326a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:39.238226 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce549edb-6b6a-4f62-a6e0-0024ee29326a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:39.239798 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce549edb-6b6a-4f62-a6e0-0024ee29326a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:39.253268 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 430/867] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:39.259999 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce549edb-6b6a-4f62-a6e0-0024ee29326a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:39.267022 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:39.312152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921b836-1aa8-438e-912b-de797e1003e2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:39.352550 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8e7c7-9d25-4b3b-8cad-5aa30bb03dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:39.353151 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8e7c7-9d25-4b3b-8cad-5aa30bb03dca None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:39.353525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8e7c7-9d25-4b3b-8cad-5aa30bb03dca None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:39.353796 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8e7c7-9d25-4b3b-8cad-5aa30bb03dca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:39.355561 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db8e7c7-9d25-4b3b-8cad-5aa30bb03dca None admin] RBAC: Adding query filter params (role.id=7554b353873942fab506de830188aba0, scope.project.id=d11f33af53374dffa61132642a7770f4, user.id=4986821ef9f84b99a835e387635aa9a8) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:39.355867 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db8e7c7-9d25-4b3b-8cad-5aa30bb03dca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:39.357295 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db8e7c7-9d25-4b3b-8cad-5aa30bb03dca None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:39.363913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 438/868] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:55:39 2022] GET /identity/v3/role_assignments?user.id=4986821ef9f84b99a835e387635aa9a8&scope.project.id=d11f33af53374dffa61132642a7770f4&role.id=7554b353873942fab506de830188aba0 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:39.393615 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce549edb-6b6a-4f62-a6e0-0024ee29326a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:39.427389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:39.434416 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191b175-0088-49bb-98ae-622185ec8da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:39.435060 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191b175-0088-49bb-98ae-622185ec8da2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:39.435332 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191b175-0088-49bb-98ae-622185ec8da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:39.435612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191b175-0088-49bb-98ae-622185ec8da2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:39.444049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191b175-0088-49bb-98ae-622185ec8da2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:39.445529 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191b175-0088-49bb-98ae-622185ec8da2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:39.459111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-6191b175-0088-49bb-98ae-622185ec8da2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 16:55:39.459111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:39.459111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:39.459111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:39.459111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:39.459111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:39.459111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:39.459111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:39.459111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:39.459111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:39.459111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:39.459111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:39.459111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:39.459111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:39.459111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:39.459111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:39.460163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:39.460163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:39.460163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:39.460163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:39.460163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:39.460163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:39.460163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:39.460163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:39.460163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:39.460163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:39.460163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:39.460163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:39.460163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:39.460163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:39.460163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:39.460163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:39.460163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:39.460163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:39.461155 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:39.461155 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:39.461155 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:39.461155 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:39.461155 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:39.461155 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:39.461155 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:39.461155 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:39.461155 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 16:55:39.461155 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:39.461155 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 431/869] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 10 16:55:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:39.468881 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8e7c7-9d25-4b3b-8cad-5aa30bb03dca None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:39.587915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8e7c7-9d25-4b3b-8cad-5aa30bb03dca None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:39.614730 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:39.628626 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a0c10-1b4e-4f71-922e-a60124f03f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:39.629247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a0c10-1b4e-4f71-922e-a60124f03f45 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:39.629576 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a0c10-1b4e-4f71-922e-a60124f03f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:39.629849 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a0c10-1b4e-4f71-922e-a60124f03f45 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:39.630606 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a0c10-1b4e-4f71-922e-a60124f03f45 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:39.630910 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a0c10-1b4e-4f71-922e-a60124f03f45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:39.632342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a0c10-1b4e-4f71-922e-a60124f03f45 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:39.655445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 439/870] 10.222.0.45 () {56 vars in 1149 bytes} [Wed Aug 10 16:55:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:39.661157 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191b175-0088-49bb-98ae-622185ec8da2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:39.770331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191b175-0088-49bb-98ae-622185ec8da2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:39.806645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2609a-af1a-4d84-98c3-1d82166a155c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:39.807302 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2609a-af1a-4d84-98c3-1d82166a155c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:39.808070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2609a-af1a-4d84-98c3-1d82166a155c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:39.808335 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2609a-af1a-4d84-98c3-1d82166a155c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:39.875589 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:39.886417 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf2609a-af1a-4d84-98c3-1d82166a155c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:39.887956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf2609a-af1a-4d84-98c3-1d82166a155c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:39.905659 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46cf3c1-3d81-4fa1-86c0-8a7635145327 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:39.905957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46cf3c1-3d81-4fa1-86c0-8a7635145327 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:39.910313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46cf3c1-3d81-4fa1-86c0-8a7635145327 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:39.910313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 440/871] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:39.913250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bdba2-52b4-4a54-ae26-bc44afc57722 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:39.913527 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bdba2-52b4-4a54-ae26-bc44afc57722 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:39.913799 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bdba2-52b4-4a54-ae26-bc44afc57722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:39.935850 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1d2bdba2-52b4-4a54-ae26-bc44afc57722 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:39.937706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1d2bdba2-52b4-4a54-ae26-bc44afc57722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:39.994944 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 432/872] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:55:39 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:55:40.006516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2609a-af1a-4d84-98c3-1d82166a155c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:40.014489 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 441/873] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:39 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:55:40.039592 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bdba2-52b4-4a54-ae26-bc44afc57722 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:40.118770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:40.133004 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2609a-af1a-4d84-98c3-1d82166a155c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:40.163012 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bdba2-52b4-4a54-ae26-bc44afc57722 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:40.177247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:40.187252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80a2cc-eb33-43a3-8b63-ab9b43b821a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:40.194566 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80a2cc-eb33-43a3-8b63-ab9b43b821a0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:40.194566 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80a2cc-eb33-43a3-8b63-ab9b43b821a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:40.194566 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80a2cc-eb33-43a3-8b63-ab9b43b821a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:40.205110 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f80a2cc-eb33-43a3-8b63-ab9b43b821a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:40.206126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94786f5-664f-4217-abf5-cdc276f63ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:40.206894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94786f5-664f-4217-abf5-cdc276f63ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:40.207218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f80a2cc-eb33-43a3-8b63-ab9b43b821a0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:40.207334 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94786f5-664f-4217-abf5-cdc276f63ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:40.207711 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94786f5-664f-4217-abf5-cdc276f63ce4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:40.208397 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94786f5-664f-4217-abf5-cdc276f63ce4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:40.210202 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94786f5-664f-4217-abf5-cdc276f63ce4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:40.221149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d94786f5-664f-4217-abf5-cdc276f63ce4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 16:55:40.221149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:40.221149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:55:40.221149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:55:40.221149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:55:40.221149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:55:40.221149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:55:40.221149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:40.221149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 16:55:40.221149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:40.221149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:55:40.221149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:40.221149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:40.221149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:40.221149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:40.221149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:40.222381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:40.222381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:40.222381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:40.222381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:40.222381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:40.222381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:40.222381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:40.222381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:55:40.222381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:55:40.222381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:55:40.222381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:55:40.222381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:40.222381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:40.222381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:40.222381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:40.222381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:40.222381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:40.223509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:40.223509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:40.223509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:40.223509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:40.223509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:40.223509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:40.223509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:40.223509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:40.223509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:40.223509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:40.223509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:55:40.223509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:40.223509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 16:55:40.223509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:40.230739 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 442/874] 10.222.0.45 () {56 vars in 1132 bytes} [Wed Aug 10 16:55:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:55:40.231431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-0f80a2cc-eb33-43a3-8b63-ab9b43b821a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:40.231431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:40.231431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:40.231431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:40.231431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:40.231431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:40.231431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:40.231431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:40.231431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:40.231431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:40.231431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:40.231431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:40.231431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:40.231431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:40.231431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:40.231431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:40.232690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:40.232690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:40.232690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:40.232690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:40.232690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:40.232690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:40.232690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:40.232690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:40.232690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:40.232690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:40.232690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:40.232690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:40.232690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:40.232690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:40.232690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:40.232690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:40.232690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:40.233806 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:40.233806 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:40.233806 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:40.234484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94786f5-664f-4217-abf5-cdc276f63ce4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:40.240165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 433/875] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:40.242214 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80a2cc-eb33-43a3-8b63-ab9b43b821a0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:40.350161 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94786f5-664f-4217-abf5-cdc276f63ce4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:40.387894 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80a2cc-eb33-43a3-8b63-ab9b43b821a0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:40.389227 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85283a25-b487-4b29-9b9f-6bc10752d37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:40.390370 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85283a25-b487-4b29-9b9f-6bc10752d37c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:40.390798 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85283a25-b487-4b29-9b9f-6bc10752d37c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:40.391901 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85283a25-b487-4b29-9b9f-6bc10752d37c None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:40.391901 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85283a25-b487-4b29-9b9f-6bc10752d37c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:40.392250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85283a25-b487-4b29-9b9f-6bc10752d37c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:40.393712 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85283a25-b487-4b29-9b9f-6bc10752d37c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:40.402682 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 443/876] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 10 16:55:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:40.405713 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85283a25-b487-4b29-9b9f-6bc10752d37c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:40.433937 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226adeb7-f434-4e1f-92f0-8835204fcda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:40.434716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226adeb7-f434-4e1f-92f0-8835204fcda0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:40.435099 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226adeb7-f434-4e1f-92f0-8835204fcda0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:40.435468 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226adeb7-f434-4e1f-92f0-8835204fcda0 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:40.436701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226adeb7-f434-4e1f-92f0-8835204fcda0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:40.437557 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226adeb7-f434-4e1f-92f0-8835204fcda0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:40.439244 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226adeb7-f434-4e1f-92f0-8835204fcda0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:40.456893 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 434/877] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:40.458240 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226adeb7-f434-4e1f-92f0-8835204fcda0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:40.458471 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:40.462804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:40.541114 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85283a25-b487-4b29-9b9f-6bc10752d37c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:40.570670 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226adeb7-f434-4e1f-92f0-8835204fcda0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:40.593138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf23ba-83a4-443e-afd7-1514b86ca5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:40.594099 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf23ba-83a4-443e-afd7-1514b86ca5a1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:40.596194 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf23ba-83a4-443e-afd7-1514b86ca5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:40.596683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf23ba-83a4-443e-afd7-1514b86ca5a1 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:40.597791 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf23ba-83a4-443e-afd7-1514b86ca5a1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:40.598580 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf23ba-83a4-443e-afd7-1514b86ca5a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:40.600913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf23ba-83a4-443e-afd7-1514b86ca5a1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:40.618545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 444/878] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 10 16:55:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:40.621002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf23ba-83a4-443e-afd7-1514b86ca5a1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:40.626562 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e2986-a0b1-486e-9cf8-8b8f9d7c796d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:40.627206 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e2986-a0b1-486e-9cf8-8b8f9d7c796d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:40.627476 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e2986-a0b1-486e-9cf8-8b8f9d7c796d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:40.627742 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e2986-a0b1-486e-9cf8-8b8f9d7c796d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:40.636975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e2986-a0b1-486e-9cf8-8b8f9d7c796d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:40.638580 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e2986-a0b1-486e-9cf8-8b8f9d7c796d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:40.651021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-821e2986-a0b1-486e-9cf8-8b8f9d7c796d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:40.651021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:40.651021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:40.651021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:40.651021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:40.651021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:40.651021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:40.651021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:40.651021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:40.651021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:40.651021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:40.651021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:40.651021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:40.651021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:40.651021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:40.652130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:40.652130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:40.652130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:40.652130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:40.652130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:40.652130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:40.652130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:40.652130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:40.652130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:40.652130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:40.652130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:40.652130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:40.652130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:40.652130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:40.652130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:40.652130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:40.652130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:40.653034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:40.653034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:40.653034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:40.653034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:40.653034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:40.653034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:40.653034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:40.653034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:40.653034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 435/879] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:55:40.658085 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e2986-a0b1-486e-9cf8-8b8f9d7c796d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:40.728147 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:40.770607 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf23ba-83a4-443e-afd7-1514b86ca5a1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:40.771955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e2986-a0b1-486e-9cf8-8b8f9d7c796d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:40.772783 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:40.808179 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce675b-ae10-423a-8a86-69d383941876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:40.809084 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce675b-ae10-423a-8a86-69d383941876 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:40.809358 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce675b-ae10-423a-8a86-69d383941876 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:40.809621 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce675b-ae10-423a-8a86-69d383941876 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:40.810314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce675b-ae10-423a-8a86-69d383941876 None admin] RBAC: Adding query filter params (interface=public, service_id=8b2ed373a16a42bebf96b168706882c3, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:40.810660 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce675b-ae10-423a-8a86-69d383941876 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:40.811822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d920a85-c3d5-4056-9e1a-ca8a021f27ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:40.812045 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce675b-ae10-423a-8a86-69d383941876 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:40.812422 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d920a85-c3d5-4056-9e1a-ca8a021f27ba None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:40.812702 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d920a85-c3d5-4056-9e1a-ca8a021f27ba None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:40.812972 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d920a85-c3d5-4056-9e1a-ca8a021f27ba None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:40.813660 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d920a85-c3d5-4056-9e1a-ca8a021f27ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:40.813963 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d920a85-c3d5-4056-9e1a-ca8a021f27ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:40.815405 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d920a85-c3d5-4056-9e1a-ca8a021f27ba None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:40.817520 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 445/880] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Aug 10 16:55:40 2022] GET /identity/v3/endpoints?service_id=8b2ed373a16a42bebf96b168706882c3&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:40.818526 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b155b-895f-4048-9744-524307374d62 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:40.818807 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b155b-895f-4048-9744-524307374d62 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:40.819070 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b155b-895f-4048-9744-524307374d62 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:40.820383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 446/881] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:40.825807 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b155b-895f-4048-9744-524307374d62 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:40.838241 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 436/882] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:40.838241 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184ddd9-d105-4d7b-8206-f03acbbe1fa3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:40.838241 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184ddd9-d105-4d7b-8206-f03acbbe1fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:40.838241 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184ddd9-d105-4d7b-8206-f03acbbe1fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:40.853157 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7184ddd9-d105-4d7b-8206-f03acbbe1fa3 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:40.854477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7184ddd9-d105-4d7b-8206-f03acbbe1fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:40.930368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 437/883] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:40 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:55:40.931145 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184ddd9-d105-4d7b-8206-f03acbbe1fa3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:40.946422 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b155b-895f-4048-9744-524307374d62 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:40.984698 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14413d0-6347-4bf9-8c94-1ef0a4f25112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:40.985399 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14413d0-6347-4bf9-8c94-1ef0a4f25112 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:40.985682 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14413d0-6347-4bf9-8c94-1ef0a4f25112 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:40.985953 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14413d0-6347-4bf9-8c94-1ef0a4f25112 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:40.986599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14413d0-6347-4bf9-8c94-1ef0a4f25112 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:40.987853 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:40.987966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14413d0-6347-4bf9-8c94-1ef0a4f25112 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:40.993984 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 447/884] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 10 16:55:40 2022] GET /identity/v3/services => generated 1268 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:55:40.993984 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14413d0-6347-4bf9-8c94-1ef0a4f25112 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:41.048945 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:41.059635 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184ddd9-d105-4d7b-8206-f03acbbe1fa3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:41.109001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc619d-5259-44a2-98fb-181ea82d78d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:41.109719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc619d-5259-44a2-98fb-181ea82d78d3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:41.109897 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc619d-5259-44a2-98fb-181ea82d78d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:41.110162 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc619d-5259-44a2-98fb-181ea82d78d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:41.110939 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dc619d-5259-44a2-98fb-181ea82d78d3 None admin] RBAC: Adding query filter params (role.id=303e0fba379a41019ecc2e119d87fb1b, scope.project.id=d11f33af53374dffa61132642a7770f4, user.id=4f9e1767ec90435f80248a2930081265) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:41.111245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dc619d-5259-44a2-98fb-181ea82d78d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:41.112700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dc619d-5259-44a2-98fb-181ea82d78d3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:41.126378 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 438/885] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:55:40 2022] GET /identity/v3/role_assignments?user.id=4f9e1767ec90435f80248a2930081265&scope.project.id=d11f33af53374dffa61132642a7770f4&role.id=303e0fba379a41019ecc2e119d87fb1b => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:41.161586 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14413d0-6347-4bf9-8c94-1ef0a4f25112 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:41.204071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06281313-a253-44bb-aff9-aa7badd9720d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:41.204644 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06281313-a253-44bb-aff9-aa7badd9720d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:41.204920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06281313-a253-44bb-aff9-aa7badd9720d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:41.205190 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06281313-a253-44bb-aff9-aa7badd9720d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:41.205781 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06281313-a253-44bb-aff9-aa7badd9720d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:41.207209 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06281313-a253-44bb-aff9-aa7badd9720d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:41.216989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-06281313-a253-44bb-aff9-aa7badd9720d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 16:55:41.216989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:41.216989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:55:41.216989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:55:41.216989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:55:41.216989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:55:41.216989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:55:41.216989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:41.216989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 16:55:41.216989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:41.216989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:55:41.216989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:41.216989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:41.216989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:41.216989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:41.216989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:41.216989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:41.218040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:41.218040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:41.218040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:41.218040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:41.218040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:41.218040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:41.218040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:55:41.218040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:55:41.218040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:55:41.218040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:55:41.218040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:41.218040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:41.218040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:41.218040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:41.218040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:41.218040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:41.218900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:41.218900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:41.218900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:41.218900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:41.218900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:41.218900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:41.218900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:41.218900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:41.218900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:41.218900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:41.218900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:55:41.218900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:41.218900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 16:55:41.218900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:41.218900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 448/886] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 10 16:55:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:41.224831 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc619d-5259-44a2-98fb-181ea82d78d3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:41.339209 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc619d-5259-44a2-98fb-181ea82d78d3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:41.356186 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:41.376359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c846bc5-1a31-4ebc-a7c6-e66a657f52fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:41.376977 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c846bc5-1a31-4ebc-a7c6-e66a657f52fe None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:41.377253 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c846bc5-1a31-4ebc-a7c6-e66a657f52fe None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:41.377520 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c846bc5-1a31-4ebc-a7c6-e66a657f52fe None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:41.378155 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c846bc5-1a31-4ebc-a7c6-e66a657f52fe None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:41.378478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c846bc5-1a31-4ebc-a7c6-e66a657f52fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:41.379851 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c846bc5-1a31-4ebc-a7c6-e66a657f52fe None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:41.384940 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 439/887] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 10 16:55:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:41.392011 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06281313-a253-44bb-aff9-aa7badd9720d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:41.501911 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:41.509209 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06281313-a253-44bb-aff9-aa7badd9720d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:41.547266 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0e22a-f6a3-425c-9f66-1deec47d7763 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:41.547547 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0e22a-f6a3-425c-9f66-1deec47d7763 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:41.547816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0e22a-f6a3-425c-9f66-1deec47d7763 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:41.549479 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 440/888] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:55:41.550384 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3b1a2-0fc6-4bc8-8f6e-5bae1d476b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:41.551250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3b1a2-0fc6-4bc8-8f6e-5bae1d476b72 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:41.551734 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3b1a2-0fc6-4bc8-8f6e-5bae1d476b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:41.552312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3b1a2-0fc6-4bc8-8f6e-5bae1d476b72 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:41.553200 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3b1a2-0fc6-4bc8-8f6e-5bae1d476b72 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:41.553936 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3b1a2-0fc6-4bc8-8f6e-5bae1d476b72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:41.554737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072ce66-dc7e-4714-9165-2497f93be3f1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:41.555014 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072ce66-dc7e-4714-9165-2497f93be3f1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:41.555281 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072ce66-dc7e-4714-9165-2497f93be3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:41.556098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3b1a2-0fc6-4bc8-8f6e-5bae1d476b72 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:41.561993 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 449/889] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 10 16:55:41 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:41.569348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3b1a2-0fc6-4bc8-8f6e-5bae1d476b72 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:41.580632 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f072ce66-dc7e-4714-9165-2497f93be3f1 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:41.581819 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f072ce66-dc7e-4714-9165-2497f93be3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:41.678403 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 441/890] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:41 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:55:41.695197 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072ce66-dc7e-4714-9165-2497f93be3f1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:41.706073 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3b1a2-0fc6-4bc8-8f6e-5bae1d476b72 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:41.743872 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72f8bb-42aa-4847-9896-caa2dd3b4337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:41.744527 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72f8bb-42aa-4847-9896-caa2dd3b4337 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:41.745759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72f8bb-42aa-4847-9896-caa2dd3b4337 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:41.746062 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72f8bb-42aa-4847-9896-caa2dd3b4337 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:41.746964 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72f8bb-42aa-4847-9896-caa2dd3b4337 None admin] RBAC: Adding query filter params (interface=public, service_id=45d1f397001c4459bef11b7e62b06508, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:41.747271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72f8bb-42aa-4847-9896-caa2dd3b4337 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:41.748728 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72f8bb-42aa-4847-9896-caa2dd3b4337 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:41.754621 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 450/891] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Aug 10 16:55:41 2022] GET /identity/v3/endpoints?service_id=45d1f397001c4459bef11b7e62b06508&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:41.759955 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72f8bb-42aa-4847-9896-caa2dd3b4337 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:41.789588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:41.812689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072ce66-dc7e-4714-9165-2497f93be3f1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:41.824913 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:41.849405 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd9baa-e7f8-4e1f-9ec7-266e0af8e6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:41.850393 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd9baa-e7f8-4e1f-9ec7-266e0af8e6f4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:41.850886 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd9baa-e7f8-4e1f-9ec7-266e0af8e6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:41.851373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd9baa-e7f8-4e1f-9ec7-266e0af8e6f4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:41.852159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddd9baa-e7f8-4e1f-9ec7-266e0af8e6f4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:41.853623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddd9baa-e7f8-4e1f-9ec7-266e0af8e6f4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:41.863221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-cddd9baa-e7f8-4e1f-9ec7-266e0af8e6f4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:55:41.863221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:41.863221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:55:41.863221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:55:41.863221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:55:41.863221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:55:41.863221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:55:41.863221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:41.863221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:55:41.863221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:41.863221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:55:41.863221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:41.863221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:41.863221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:41.863221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:41.863221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:41.863221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:41.864642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:41.864642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:41.864642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:41.864642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:41.864642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:41.864642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:41.864642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:55:41.864642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:55:41.864642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:55:41.864642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:55:41.864642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:41.864642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:41.864642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:41.864642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:41.864642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:41.864642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:41.865485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:41.865485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:41.865485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:41.865485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:41.865485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:41.865485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:41.865485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:41.865485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:41.865485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:41.865485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:41.865485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:55:41.865485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:41.865485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:55:41.865485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:41.869363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 442/892] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 10 16:55:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:55:41.870919 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd9baa-e7f8-4e1f-9ec7-266e0af8e6f4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:41.890250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72f8bb-42aa-4847-9896-caa2dd3b4337 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:41.926896 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c98128-8f85-47e5-86d6-bffda42b8903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:41.927900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c98128-8f85-47e5-86d6-bffda42b8903 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:41.928359 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c98128-8f85-47e5-86d6-bffda42b8903 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:41.928806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c98128-8f85-47e5-86d6-bffda42b8903 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:41.929581 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c98128-8f85-47e5-86d6-bffda42b8903 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:41.931182 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c98128-8f85-47e5-86d6-bffda42b8903 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:41.938162 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 451/893] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 10 16:55:41 2022] GET /identity/v3/services => generated 1268 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:55:41.983732 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd9baa-e7f8-4e1f-9ec7-266e0af8e6f4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:42.022730 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8381c06-a187-4287-bd14-c5c5c182a05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:42.023262 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8381c06-a187-4287-bd14-c5c5c182a05c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:42.023537 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8381c06-a187-4287-bd14-c5c5c182a05c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:42.024069 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8381c06-a187-4287-bd14-c5c5c182a05c None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:42.024716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8381c06-a187-4287-bd14-c5c5c182a05c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:42.025015 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8381c06-a187-4287-bd14-c5c5c182a05c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:42.026490 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8381c06-a187-4287-bd14-c5c5c182a05c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:42.036967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 443/894] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 10 16:55:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:42.038528 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c98128-8f85-47e5-86d6-bffda42b8903 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:42.139208 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:42.155502 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c98128-8f85-47e5-86d6-bffda42b8903 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:42.192509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfa54e-75d5-4798-ae60-4faddec11575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:42.193114 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfa54e-75d5-4798-ae60-4faddec11575 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:42.193386 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfa54e-75d5-4798-ae60-4faddec11575 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:42.193654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfa54e-75d5-4798-ae60-4faddec11575 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:42.194280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cfa54e-75d5-4798-ae60-4faddec11575 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:42.194599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cfa54e-75d5-4798-ae60-4faddec11575 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:42.195969 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cfa54e-75d5-4798-ae60-4faddec11575 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:42.201973 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 452/895] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 10 16:55:42 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:43.173279 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d14c07-5780-4433-b9f8-f7ba9f7a9774 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:43.173809 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d14c07-5780-4433-b9f8-f7ba9f7a9774 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:43.173809 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d14c07-5780-4433-b9f8-f7ba9f7a9774 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:43.175459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 444/896] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:43.180786 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c1c23-9b93-4f77-8bc8-bcfa9a1ae07a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:43.181193 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c1c23-9b93-4f77-8bc8-bcfa9a1ae07a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:43.181360 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c1c23-9b93-4f77-8bc8-bcfa9a1ae07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:43.200438 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-109c1c23-9b93-4f77-8bc8-bcfa9a1ae07a None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:43.201636 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-109c1c23-9b93-4f77-8bc8-bcfa9a1ae07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:43.273360 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 453/897] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:55:43.299536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d14c07-5780-4433-b9f8-f7ba9f7a9774 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:43.346036 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:43.410410 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d14c07-5780-4433-b9f8-f7ba9f7a9774 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:43.416092 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627cabd-8597-46c5-89a6-8bdfdfb9a346 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:43.417227 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627cabd-8597-46c5-89a6-8bdfdfb9a346 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:43.417708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627cabd-8597-46c5-89a6-8bdfdfb9a346 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:43.419280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 454/898] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:55:43.424629 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0fcfd-b2d6-402c-b1e2-08aecc04066f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:43.424856 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0fcfd-b2d6-402c-b1e2-08aecc04066f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:43.425101 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0fcfd-b2d6-402c-b1e2-08aecc04066f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:43.445041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bda0fcfd-b2d6-402c-b1e2-08aecc04066f None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:43.446786 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bda0fcfd-b2d6-402c-b1e2-08aecc04066f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:43.447516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556ec93-4388-4666-951a-dbcc063c4b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:43.449333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556ec93-4388-4666-951a-dbcc063c4b3f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:43.449607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556ec93-4388-4666-951a-dbcc063c4b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:43.449872 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556ec93-4388-4666-951a-dbcc063c4b3f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:43.450480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556ec93-4388-4666-951a-dbcc063c4b3f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:43.452418 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556ec93-4388-4666-951a-dbcc063c4b3f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:43.460770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-8556ec93-4388-4666-951a-dbcc063c4b3f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 16:55:43.460770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:43.460770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:55:43.460770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:55:43.460770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:55:43.460770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:55:43.460770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:55:43.460770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:43.460770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 16:55:43.460770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:43.460770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:55:43.460770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:43.460770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:43.460770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:43.460770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:43.460770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:43.461757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:43.461757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:43.461757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:43.461757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:43.461757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:43.461757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:43.461757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:43.461757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:55:43.461757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:55:43.461757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:55:43.461757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:55:43.461757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:43.461757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:43.461757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:43.461757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:43.461757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:43.461757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:43.463065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:43.463065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:43.463065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:43.463065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:43.463065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:43.463065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:43.463065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:43.463065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:43.463065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:43.463065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:43.463065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:55:43.463065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:43.463065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 16:55:43.463065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:43.468047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 445/899] 10.222.0.45 () {56 vars in 1132 bytes} [Wed Aug 10 16:55:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:43.469162 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556ec93-4388-4666-951a-dbcc063c4b3f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:43.524060 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 455/900] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:55:43.548592 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0fcfd-b2d6-402c-b1e2-08aecc04066f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:43.560626 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:43.583206 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556ec93-4388-4666-951a-dbcc063c4b3f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:43.584921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:43.628229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f8e9d-4926-4b55-bd33-07541c9bb2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:43.629180 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f8e9d-4926-4b55-bd33-07541c9bb2c6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:43.629598 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f8e9d-4926-4b55-bd33-07541c9bb2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:43.630132 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f8e9d-4926-4b55-bd33-07541c9bb2c6 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:43.630990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f8e9d-4926-4b55-bd33-07541c9bb2c6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:43.632458 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f8e9d-4926-4b55-bd33-07541c9bb2c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:43.634137 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f8e9d-4926-4b55-bd33-07541c9bb2c6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:43.641737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 446/901] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 10 16:55:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:43.647453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f8e9d-4926-4b55-bd33-07541c9bb2c6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:43.656397 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0fcfd-b2d6-402c-b1e2-08aecc04066f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:43.691901 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b850827-049f-411c-8dab-a1ce1cafa84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:43.692478 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b850827-049f-411c-8dab-a1ce1cafa84d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:43.692754 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b850827-049f-411c-8dab-a1ce1cafa84d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:43.693082 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b850827-049f-411c-8dab-a1ce1cafa84d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:43.710416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b850827-049f-411c-8dab-a1ce1cafa84d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:43.710416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4b850827-049f-411c-8dab-a1ce1cafa84d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 16:55:43.710416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:43.710416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:43.710416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:43.710416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:43.710416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:43.710416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:43.710416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:43.710416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:43.710416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:43.710416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:43.710416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:43.710416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:43.711338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:43.711338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:43.711338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:43.711338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:43.711338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:43.711338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:43.711338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:43.711338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:43.711338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:43.711338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:43.711338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:43.711338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:43.711338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:43.711338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:43.711338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:43.711338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:43.711338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:43.712228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:43.712228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:43.712228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:43.712228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:43.712228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:43.712228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:43.712228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:43.712228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:43.712228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:43.712228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:43.712228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:43.712228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:43.712228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 16:55:43.712228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:43.712228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 456/902] 10.222.0.45 () {56 vars in 1120 bytes} [Wed Aug 10 16:55:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:43.720940 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b850827-049f-411c-8dab-a1ce1cafa84d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:43.768903 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f8e9d-4926-4b55-bd33-07541c9bb2c6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:43.814501 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9983a-81f0-4c37-b3a8-59b60e421026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:43.815150 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9983a-81f0-4c37-b3a8-59b60e421026 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:43.815451 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9983a-81f0-4c37-b3a8-59b60e421026 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:43.815683 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9983a-81f0-4c37-b3a8-59b60e421026 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:43.816317 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e9983a-81f0-4c37-b3a8-59b60e421026 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:43.816612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e9983a-81f0-4c37-b3a8-59b60e421026 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:43.818047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e9983a-81f0-4c37-b3a8-59b60e421026 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:43.827827 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 447/903] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 10 16:55:43 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:43.828938 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9983a-81f0-4c37-b3a8-59b60e421026 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:43.834542 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:43.862457 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:43.882171 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b850827-049f-411c-8dab-a1ce1cafa84d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:43.939188 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9983a-81f0-4c37-b3a8-59b60e421026 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:43.947149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1c185-3b76-4d8c-928e-e885e187e698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:43.948142 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1c185-3b76-4d8c-928e-e885e187e698 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:43.948424 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1c185-3b76-4d8c-928e-e885e187e698 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:43.948877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1c185-3b76-4d8c-928e-e885e187e698 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:43.949560 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce1c185-3b76-4d8c-928e-e885e187e698 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:43.949862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce1c185-3b76-4d8c-928e-e885e187e698 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:43.951644 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce1c185-3b76-4d8c-928e-e885e187e698 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:43.966456 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 457/904] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Aug 10 16:55:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:43.969014 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1c185-3b76-4d8c-928e-e885e187e698 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:43.982004 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0cf23-8c30-4455-95a2-5ec8cdc2b043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:43.982657 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0cf23-8c30-4455-95a2-5ec8cdc2b043 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:43.982951 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0cf23-8c30-4455-95a2-5ec8cdc2b043 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:43.983201 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0cf23-8c30-4455-95a2-5ec8cdc2b043 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:43.983856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0cf23-8c30-4455-95a2-5ec8cdc2b043 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:43.985294 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0cf23-8c30-4455-95a2-5ec8cdc2b043 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:44.066069 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 448/905] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 16:55:43 2022] POST /identity/v3/endpoints => generated 347 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:44.067741 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336d44f-6a39-4e10-87d9-d7b86baf6b56 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:44.068093 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336d44f-6a39-4e10-87d9-d7b86baf6b56 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:44.068440 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336d44f-6a39-4e10-87d9-d7b86baf6b56 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:44.070394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 449/906] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:44.076337 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4006223-6a01-4542-897c-9620f9125bdd None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:44.077103 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4006223-6a01-4542-897c-9620f9125bdd None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:44.077372 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4006223-6a01-4542-897c-9620f9125bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:44.092196 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1c185-3b76-4d8c-928e-e885e187e698 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:44.097066 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a4006223-6a01-4542-897c-9620f9125bdd None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:44.098449 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a4006223-6a01-4542-897c-9620f9125bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:44.130058 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:44.143871 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67faa65-8572-477c-b8f1-bb8c8abcd232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:44.144838 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67faa65-8572-477c-b8f1-bb8c8abcd232 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:44.145310 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67faa65-8572-477c-b8f1-bb8c8abcd232 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:44.145741 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67faa65-8572-477c-b8f1-bb8c8abcd232 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:44.162664 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67faa65-8572-477c-b8f1-bb8c8abcd232 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:44.164756 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67faa65-8572-477c-b8f1-bb8c8abcd232 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:44.183948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c67faa65-8572-477c-b8f1-bb8c8abcd232 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:44.183948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:44.183948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:44.183948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:44.183948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:44.183948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:44.183948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:44.183948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:44.183948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:44.183948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:44.183948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:44.183948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:44.183948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:44.183948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:44.183948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:44.183948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:44.185445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:44.185445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:44.185445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:44.185445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:44.185445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:44.185445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:44.185445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:44.185445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:44.185445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:44.185445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:44.185445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:44.185445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:44.185445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:44.185445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:44.185445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:44.185445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:44.185445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:44.186785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:44.186785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:44.186785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:44.188868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 458/907] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:55:44.191690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 450/908] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:55:44.199806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67faa65-8572-477c-b8f1-bb8c8abcd232 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:44.218113 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4006223-6a01-4542-897c-9620f9125bdd None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:44.242836 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:44.324736 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67faa65-8572-477c-b8f1-bb8c8abcd232 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:44.344326 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4006223-6a01-4542-897c-9620f9125bdd None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:44.363456 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d979d-e876-4e1e-88ee-d497d157ca57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:44.364055 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d979d-e876-4e1e-88ee-d497d157ca57 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:44.364330 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d979d-e876-4e1e-88ee-d497d157ca57 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:44.364597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d979d-e876-4e1e-88ee-d497d157ca57 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:44.365228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d979d-e876-4e1e-88ee-d497d157ca57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:44.365526 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d979d-e876-4e1e-88ee-d497d157ca57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:44.366945 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d979d-e876-4e1e-88ee-d497d157ca57 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:44.380049 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 459/909] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:44.381278 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3959677-9c23-489d-aff4-f505ff2c782b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:44.381556 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3959677-9c23-489d-aff4-f505ff2c782b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:44.381818 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3959677-9c23-489d-aff4-f505ff2c782b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:44.383217 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 460/910] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:44.387385 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3959677-9c23-489d-aff4-f505ff2c782b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:44.390341 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6be43-94b6-4549-bf3f-ea929cd7eccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:44.391177 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6be43-94b6-4549-bf3f-ea929cd7eccc None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:44.391449 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6be43-94b6-4549-bf3f-ea929cd7eccc None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:44.391714 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6be43-94b6-4549-bf3f-ea929cd7eccc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:44.392296 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac6be43-94b6-4549-bf3f-ea929cd7eccc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:44.393674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac6be43-94b6-4549-bf3f-ea929cd7eccc None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:44.403374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-dac6be43-94b6-4549-bf3f-ea929cd7eccc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 16:55:44.403374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:44.403374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:55:44.403374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:55:44.403374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:55:44.403374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:55:44.403374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:55:44.403374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:44.403374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 16:55:44.403374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:44.403374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:55:44.403374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:44.403374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:44.403374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:44.403374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:44.403374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:44.403374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:44.404431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:44.404431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:44.404431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:44.404431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:44.404431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:44.404431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:44.404431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:55:44.404431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:55:44.404431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:55:44.404431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:55:44.404431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:44.404431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:44.404431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:44.404431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:44.404431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:44.404431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:44.405308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:44.405308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:44.405308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:44.405308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:44.405308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:44.405308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:44.405308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:44.405308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:44.405308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:44.405308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:44.405308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:55:44.405308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:44.405308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 16:55:44.405308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:44.405308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 451/911] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 10 16:55:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:55:44.406113 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa9a06-a5cc-4659-8a1a-721bfbbadb82 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:44.406113 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa9a06-a5cc-4659-8a1a-721bfbbadb82 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:44.406245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa9a06-a5cc-4659-8a1a-721bfbbadb82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:44.425933 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f3aa9a06-a5cc-4659-8a1a-721bfbbadb82 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:44.427142 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f3aa9a06-a5cc-4659-8a1a-721bfbbadb82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:44.442945 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:44.503996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3959677-9c23-489d-aff4-f505ff2c782b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:44.510504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 452/912] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:55:44.511435 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa9a06-a5cc-4659-8a1a-721bfbbadb82 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:44.526925 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:44.547120 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb05f6-1827-4fa5-b491-6bb104d18f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:44.547750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb05f6-1827-4fa5-b491-6bb104d18f3e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:44.548023 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb05f6-1827-4fa5-b491-6bb104d18f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:44.548287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb05f6-1827-4fa5-b491-6bb104d18f3e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:44.556039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb05f6-1827-4fa5-b491-6bb104d18f3e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:44.558527 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb05f6-1827-4fa5-b491-6bb104d18f3e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:44.570642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5bfb05f6-1827-4fa5-b491-6bb104d18f3e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 16:55:44.570642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:44.570642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:44.570642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:44.570642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:44.570642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:44.570642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:44.570642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:44.570642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:44.570642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:44.570642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:44.570642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:44.570642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:44.570642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:44.570642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:44.570642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:44.571649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:44.571649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:44.571649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:44.571649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:44.571649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:44.571649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:44.571649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:44.571649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:44.571649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:44.571649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:44.571649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:44.571649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:44.571649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:44.571649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:44.571649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:44.571649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:44.571649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:44.571649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:44.572593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:44.572593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:44.572593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:44.572593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:44.572593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:44.572593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:44.572593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:44.572593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:44.572593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 16:55:44.572593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:44.576815 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 461/913] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 10 16:55:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:44.577664 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb05f6-1827-4fa5-b491-6bb104d18f3e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:44.634713 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa9a06-a5cc-4659-8a1a-721bfbbadb82 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:44.672226 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b6365-7fa8-46de-af83-328cd2888430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:44.672826 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b6365-7fa8-46de-af83-328cd2888430 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:44.673093 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b6365-7fa8-46de-af83-328cd2888430 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:44.673363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b6365-7fa8-46de-af83-328cd2888430 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:44.674016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b6365-7fa8-46de-af83-328cd2888430 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:44.674309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b6365-7fa8-46de-af83-328cd2888430 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:44.675692 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b6365-7fa8-46de-af83-328cd2888430 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:44.680490 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 453/914] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 10 16:55:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:44.681256 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b6365-7fa8-46de-af83-328cd2888430 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:44.696493 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:44.700266 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb05f6-1827-4fa5-b491-6bb104d18f3e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:44.736913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ced6b5-78db-4f3e-9f25-dd089d67ccb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:44.737680 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ced6b5-78db-4f3e-9f25-dd089d67ccb7 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:44.738299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ced6b5-78db-4f3e-9f25-dd089d67ccb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:44.738782 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ced6b5-78db-4f3e-9f25-dd089d67ccb7 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:44.739817 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ced6b5-78db-4f3e-9f25-dd089d67ccb7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:44.741350 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ced6b5-78db-4f3e-9f25-dd089d67ccb7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:44.765794 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:44.790075 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b6365-7fa8-46de-af83-328cd2888430 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:44.830457 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 462/915] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 10 16:55:44 2022] POST /identity/v3/services => generated 241 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:44.830457 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ced6b5-78db-4f3e-9f25-dd089d67ccb7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:44.843619 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771af35-ad51-4e50-bdb1-650dc2a63a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:44.844268 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771af35-ad51-4e50-bdb1-650dc2a63a67 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:44.844542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771af35-ad51-4e50-bdb1-650dc2a63a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:44.844804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771af35-ad51-4e50-bdb1-650dc2a63a67 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:44.845666 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0771af35-ad51-4e50-bdb1-650dc2a63a67 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:44.845996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0771af35-ad51-4e50-bdb1-650dc2a63a67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:44.847542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0771af35-ad51-4e50-bdb1-650dc2a63a67 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:44.868282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 454/916] 10.222.0.45 () {56 vars in 1149 bytes} [Wed Aug 10 16:55:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:44.875005 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771af35-ad51-4e50-bdb1-650dc2a63a67 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:44.971217 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ced6b5-78db-4f3e-9f25-dd089d67ccb7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:45.002611 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771af35-ad51-4e50-bdb1-650dc2a63a67 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:45.021567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d291a3-465e-4bcf-b7ec-ca81f64a6e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:45.022336 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d291a3-465e-4bcf-b7ec-ca81f64a6e8f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:45.022785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d291a3-465e-4bcf-b7ec-ca81f64a6e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:45.023169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d291a3-465e-4bcf-b7ec-ca81f64a6e8f None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:45.024168 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d291a3-465e-4bcf-b7ec-ca81f64a6e8f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:45.024550 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d291a3-465e-4bcf-b7ec-ca81f64a6e8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:45.026048 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d291a3-465e-4bcf-b7ec-ca81f64a6e8f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:45.031688 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 463/917] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 10 16:55:44 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:45.037416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d291a3-465e-4bcf-b7ec-ca81f64a6e8f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:45.038132 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:45.049183 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb2325-f449-493b-9b2a-b667f0443939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:45.053010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb2325-f449-493b-9b2a-b667f0443939 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:45.053010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb2325-f449-493b-9b2a-b667f0443939 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:45.053010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb2325-f449-493b-9b2a-b667f0443939 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:45.068992 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:45.136079 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fb2325-f449-493b-9b2a-b667f0443939 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:45.138834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fb2325-f449-493b-9b2a-b667f0443939 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:45.164369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d291a3-465e-4bcf-b7ec-ca81f64a6e8f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:45.213980 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36514941-1927-41a9-aef5-87b30daab38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:45.215331 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36514941-1927-41a9-aef5-87b30daab38f None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:45.215609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36514941-1927-41a9-aef5-87b30daab38f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:45.215875 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36514941-1927-41a9-aef5-87b30daab38f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:45.216538 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36514941-1927-41a9-aef5-87b30daab38f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:45.217902 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36514941-1927-41a9-aef5-87b30daab38f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:45.263032 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 455/918] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:55:44 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 16:55:45.268891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb2325-f449-493b-9b2a-b667f0443939 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:45.299141 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:45.350489 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 464/919] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 16:55:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:55:45.390782 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb2325-f449-493b-9b2a-b667f0443939 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:45.439092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e90dad-e3ea-4227-8f0e-da7c66675715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:45.439753 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e90dad-e3ea-4227-8f0e-da7c66675715 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:45.440028 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e90dad-e3ea-4227-8f0e-da7c66675715 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:45.440293 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e90dad-e3ea-4227-8f0e-da7c66675715 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:45.452820 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e90dad-e3ea-4227-8f0e-da7c66675715 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:45.454470 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e90dad-e3ea-4227-8f0e-da7c66675715 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:45.471887 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-46e90dad-e3ea-4227-8f0e-da7c66675715 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:45.471887 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:45.471887 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:45.471887 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:45.471887 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:45.471887 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:45.471887 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:45.471887 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:45.471887 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:45.471887 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:45.471887 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:45.471887 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:45.471887 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:45.471887 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:45.471887 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:45.471887 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:45.472912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:45.472912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:45.472912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:45.472912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:45.472912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:45.472912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:45.472912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:45.472912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:45.472912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:45.472912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:45.472912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:45.472912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:45.472912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:45.472912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:45.472912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:45.472912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:45.472912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:45.473845 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:45.473845 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:45.473845 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:45.473845 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 456/920] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:55:45.479006 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36514941-1927-41a9-aef5-87b30daab38f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:45.569263 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:45.589517 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36514941-1927-41a9-aef5-87b30daab38f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:45.630229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70537a12-2813-4df6-a70c-46ff2d8a6c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:45.639591 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70537a12-2813-4df6-a70c-46ff2d8a6c9c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:45.639591 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70537a12-2813-4df6-a70c-46ff2d8a6c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:45.639591 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70537a12-2813-4df6-a70c-46ff2d8a6c9c None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:45.639591 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70537a12-2813-4df6-a70c-46ff2d8a6c9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:45.639591 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70537a12-2813-4df6-a70c-46ff2d8a6c9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:45.639591 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70537a12-2813-4df6-a70c-46ff2d8a6c9c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:45.650484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 465/921] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:45.653291 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e90dad-e3ea-4227-8f0e-da7c66675715 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:45.737517 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:45.762275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e90dad-e3ea-4227-8f0e-da7c66675715 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:45.798082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd365e74-c201-4ddf-88dc-6f616511a9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:45.798746 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd365e74-c201-4ddf-88dc-6f616511a9ac None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:45.799016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd365e74-c201-4ddf-88dc-6f616511a9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:45.799287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd365e74-c201-4ddf-88dc-6f616511a9ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:45.806473 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd365e74-c201-4ddf-88dc-6f616511a9ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:45.808517 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd365e74-c201-4ddf-88dc-6f616511a9ac None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:45.819975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-dd365e74-c201-4ddf-88dc-6f616511a9ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:45.819975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:45.819975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:45.819975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:45.819975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:45.819975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:45.819975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:45.819975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:45.819975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:45.819975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:45.819975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:45.819975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:45.819975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:45.819975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:45.819975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:45.821082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:45.821082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:45.821082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:45.821082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:45.821082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:45.821082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:45.821082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:45.821082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:45.821082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:45.821082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:45.821082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:45.821082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:45.821082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:45.821082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:45.821082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:45.821082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:45.821082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:45.822279 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:45.822279 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:45.822279 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:45.822279 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:45.822279 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:45.822279 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:45.822279 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:45.822279 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:45.822279 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 457/922] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:45.828884 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70537a12-2813-4df6-a70c-46ff2d8a6c9c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:45.942253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70537a12-2813-4df6-a70c-46ff2d8a6c9c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:45.979287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc7fd2f-0da0-40ef-bf44-41147f83aed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:45.980198 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7fd2f-0da0-40ef-bf44-41147f83aed3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:45.980619 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7fd2f-0da0-40ef-bf44-41147f83aed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:45.981055 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7fd2f-0da0-40ef-bf44-41147f83aed3 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:45.981896 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc7fd2f-0da0-40ef-bf44-41147f83aed3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:45.982342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc7fd2f-0da0-40ef-bf44-41147f83aed3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:45.983922 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc7fd2f-0da0-40ef-bf44-41147f83aed3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:45.993117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:46.001615 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 466/923] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:46.004094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd365e74-c201-4ddf-88dc-6f616511a9ac None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:46.122256 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd365e74-c201-4ddf-88dc-6f616511a9ac None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:46.158791 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aec4ed-42bf-4f82-90e2-70fd316c735c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:46.159545 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aec4ed-42bf-4f82-90e2-70fd316c735c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:46.159889 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aec4ed-42bf-4f82-90e2-70fd316c735c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:46.160224 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aec4ed-42bf-4f82-90e2-70fd316c735c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:46.161019 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aec4ed-42bf-4f82-90e2-70fd316c735c None admin] RBAC: Adding query filter params (role.id=c37c16d90ff44d80991a2e4bdb50432c, scope.project.id=d11f33af53374dffa61132642a7770f4, user.id=4f9e1767ec90435f80248a2930081265) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:46.161392 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aec4ed-42bf-4f82-90e2-70fd316c735c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:46.162922 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aec4ed-42bf-4f82-90e2-70fd316c735c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:46.166019 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:46.169674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 458/924] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:55:46 2022] GET /identity/v3/role_assignments?user.id=4f9e1767ec90435f80248a2930081265&scope.project.id=d11f33af53374dffa61132642a7770f4&role.id=c37c16d90ff44d80991a2e4bdb50432c => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:46.257510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddf6c0-3892-4c9d-9510-14d2a270a349 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:46.258042 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddf6c0-3892-4c9d-9510-14d2a270a349 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:46.258042 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddf6c0-3892-4c9d-9510-14d2a270a349 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:46.259788 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 467/925] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:46.266341 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b738444-c667-450b-9519-b22da093dc72 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:46.266877 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b738444-c667-450b-9519-b22da093dc72 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:46.266967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b738444-c667-450b-9519-b22da093dc72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:46.286512 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5b738444-c667-450b-9519-b22da093dc72 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:46.287745 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5b738444-c667-450b-9519-b22da093dc72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:46.361944 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 459/926] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:55:46.384982 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddf6c0-3892-4c9d-9510-14d2a270a349 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:46.492894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddf6c0-3892-4c9d-9510-14d2a270a349 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:46.529677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f1f67-0548-4249-b1d8-8a4259809ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:46.530698 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f1f67-0548-4249-b1d8-8a4259809ebc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:46.531216 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f1f67-0548-4249-b1d8-8a4259809ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:46.531697 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f1f67-0548-4249-b1d8-8a4259809ebc None admin] PATH_INFO: `/v3/services/compute` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:46.532489 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265f1f67-0548-4249-b1d8-8a4259809ebc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:46.534069 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265f1f67-0548-4249-b1d8-8a4259809ebc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:46.544694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-265f1f67-0548-4249-b1d8-8a4259809ebc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 16:55:46.544694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:46.544694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:55:46.544694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:55:46.544694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:55:46.544694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:55:46.544694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:55:46.544694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:46.544694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 16:55:46.544694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:46.544694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:55:46.544694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:46.544694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:46.544694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:46.544694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:46.544694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:46.544694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:46.545774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:46.545774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:46.545774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:46.545774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:46.545774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:46.545774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:46.545774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:55:46.545774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:55:46.545774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:55:46.545774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:55:46.545774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:46.545774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:46.545774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:46.545774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:46.545774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:46.545774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:46.546663 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:46.546663 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:46.546663 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:46.546663 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:46.546663 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:46.546663 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:46.546663 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:46.546663 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:46.546663 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:46.546663 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:46.546663 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:55:46.546663 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:46.546663 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 16:55:46.546663 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:46.549740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 468/927] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:46.555556 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b738444-c667-450b-9519-b22da093dc72 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:46.666833 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b738444-c667-450b-9519-b22da093dc72 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:46.696954 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:46.703845 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ca5cf-fe35-4579-98cc-940f02d1a732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:46.704454 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ca5cf-fe35-4579-98cc-940f02d1a732 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:46.704736 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ca5cf-fe35-4579-98cc-940f02d1a732 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:46.705005 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ca5cf-fe35-4579-98cc-940f02d1a732 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:46.705646 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ca5cf-fe35-4579-98cc-940f02d1a732 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:46.705947 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ca5cf-fe35-4579-98cc-940f02d1a732 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:46.707352 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ca5cf-fe35-4579-98cc-940f02d1a732 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:46.712699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 460/928] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 10 16:55:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:46.719341 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f1f67-0548-4249-b1d8-8a4259809ebc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:46.786324 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:46.828510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f1f67-0548-4249-b1d8-8a4259809ebc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:46.865439 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362bd729-6d95-49c1-815f-fd15f11fee40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:46.866053 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bd729-6d95-49c1-815f-fd15f11fee40 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:46.866330 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bd729-6d95-49c1-815f-fd15f11fee40 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:46.866630 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bd729-6d95-49c1-815f-fd15f11fee40 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:46.867256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362bd729-6d95-49c1-815f-fd15f11fee40 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:46.867555 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362bd729-6d95-49c1-815f-fd15f11fee40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:46.869031 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362bd729-6d95-49c1-815f-fd15f11fee40 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:46.874113 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 469/929] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 10 16:55:46 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:47.067947 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0dafb-786b-439d-98f2-d64219f09e45 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:47.068284 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0dafb-786b-439d-98f2-d64219f09e45 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:47.068459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0dafb-786b-439d-98f2-d64219f09e45 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:47.069953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 461/930] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:47.075672 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd1d61-e816-4c6f-a27c-fa24f1376a8e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:47.075959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd1d61-e816-4c6f-a27c-fa24f1376a8e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:47.076233 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd1d61-e816-4c6f-a27c-fa24f1376a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:47.096762 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-07fd1d61-e816-4c6f-a27c-fa24f1376a8e None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:47.098281 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-07fd1d61-e816-4c6f-a27c-fa24f1376a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:47.174517 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 470/931] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:55:47.192186 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0dafb-786b-439d-98f2-d64219f09e45 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:47.300868 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0dafb-786b-439d-98f2-d64219f09e45 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:47.336496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60b414-bc0c-4d68-bb0e-18686d332cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:47.337146 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60b414-bc0c-4d68-bb0e-18686d332cb4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:47.337419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60b414-bc0c-4d68-bb0e-18686d332cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:47.337689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60b414-bc0c-4d68-bb0e-18686d332cb4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:47.338312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60b414-bc0c-4d68-bb0e-18686d332cb4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:47.339773 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60b414-bc0c-4d68-bb0e-18686d332cb4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:47.349594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-5f60b414-bc0c-4d68-bb0e-18686d332cb4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:55:47.349594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:47.349594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:55:47.349594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:55:47.349594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:55:47.349594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:55:47.349594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:55:47.349594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:47.349594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:55:47.349594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:47.349594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:55:47.349594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:47.349594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:47.349594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:47.349594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:47.349594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:47.349594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:47.350700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:47.350700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:47.350700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:47.350700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:47.350700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:47.350700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:47.350700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:55:47.350700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:55:47.350700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:55:47.350700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:55:47.350700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:47.350700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:47.350700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:47.350700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:47.350700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:47.350700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:47.351810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:47.351810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:47.351810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:47.351810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:47.351810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:47.351810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:47.351810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:47.351810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:47.351810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:47.351810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:47.351810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:55:47.351810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:47.351810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:55:47.351810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:47.351810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 462/932] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 10 16:55:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:47.359756 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd1d61-e816-4c6f-a27c-fa24f1376a8e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:47.425249 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:47.467980 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd1d61-e816-4c6f-a27c-fa24f1376a8e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:47.508769 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00494f90-0674-4702-8ffb-800cac43c8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:47.509440 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00494f90-0674-4702-8ffb-800cac43c8b8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:47.509757 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00494f90-0674-4702-8ffb-800cac43c8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:47.510124 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00494f90-0674-4702-8ffb-800cac43c8b8 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:47.511085 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00494f90-0674-4702-8ffb-800cac43c8b8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:47.511391 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00494f90-0674-4702-8ffb-800cac43c8b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:47.512829 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00494f90-0674-4702-8ffb-800cac43c8b8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:47.517376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12581a36-d086-4daf-8fdb-808724756bac None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:47.517682 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12581a36-d086-4daf-8fdb-808724756bac None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:47.517934 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12581a36-d086-4daf-8fdb-808724756bac None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:47.518479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 471/933] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 10 16:55:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:47.519388 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 463/934] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:47.524685 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00494f90-0674-4702-8ffb-800cac43c8b8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:47.525118 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eba633-1c2c-418e-889a-b1ccce027493 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:47.525420 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eba633-1c2c-418e-889a-b1ccce027493 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:47.525685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eba633-1c2c-418e-889a-b1ccce027493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:47.545908 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b6eba633-1c2c-418e-889a-b1ccce027493 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:47.547144 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b6eba633-1c2c-418e-889a-b1ccce027493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:47.610435 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:47.630510 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 464/935] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:55:47.652001 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00494f90-0674-4702-8ffb-800cac43c8b8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:47.653590 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eba633-1c2c-418e-889a-b1ccce027493 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:47.686073 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:47.702787 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1025e-87f4-4f4d-9e12-87941f3580a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:47.707771 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1025e-87f4-4f4d-9e12-87941f3580a1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:47.707771 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1025e-87f4-4f4d-9e12-87941f3580a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:47.707771 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1025e-87f4-4f4d-9e12-87941f3580a1 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:47.707771 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c1025e-87f4-4f4d-9e12-87941f3580a1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:47.707771 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c1025e-87f4-4f4d-9e12-87941f3580a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:47.707771 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c1025e-87f4-4f4d-9e12-87941f3580a1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:47.716329 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 472/936] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 10 16:55:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:47.722152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1025e-87f4-4f4d-9e12-87941f3580a1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:47.777755 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eba633-1c2c-418e-889a-b1ccce027493 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:47.816986 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893a981-8947-4ba3-bd30-abb9bb3d3076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:47.817982 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893a981-8947-4ba3-bd30-abb9bb3d3076 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:47.818617 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893a981-8947-4ba3-bd30-abb9bb3d3076 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:47.819107 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893a981-8947-4ba3-bd30-abb9bb3d3076 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:47.820010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c893a981-8947-4ba3-bd30-abb9bb3d3076 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:47.821759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c893a981-8947-4ba3-bd30-abb9bb3d3076 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:47.825702 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: WARNING keystone.server.flask.common [None req-c893a981-8947-4ba3-bd30-abb9bb3d3076 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 10 16:55:47.828966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1025e-87f4-4f4d-9e12-87941f3580a1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:47.863952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e854fe0-b22c-469a-8ef8-70bfe6678bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:47.864581 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e854fe0-b22c-469a-8ef8-70bfe6678bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:47.864863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e854fe0-b22c-469a-8ef8-70bfe6678bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:47.865126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e854fe0-b22c-469a-8ef8-70bfe6678bc5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:47.865810 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e854fe0-b22c-469a-8ef8-70bfe6678bc5 None admin] RBAC: Adding query filter params (interface=public, service_id=25c79f8c874d4fd59f7058392fe637c8, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:47.866108 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e854fe0-b22c-469a-8ef8-70bfe6678bc5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:47.867530 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e854fe0-b22c-469a-8ef8-70bfe6678bc5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:47.873561 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 473/937] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Aug 10 16:55:47 2022] GET /identity/v3/endpoints?service_id=25c79f8c874d4fd59f7058392fe637c8&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:47.879372 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e854fe0-b22c-469a-8ef8-70bfe6678bc5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:47.879916 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.sql.core [None req-c893a981-8947-4ba3-bd30-abb9bb3d3076 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 16:55:47.879916 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 10 16:55:47.879916 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [parameters: {'id': '6d78a27972e44a3790dc30c38b0759a8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 10 16:55:47.879916 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81612) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 16:55:47.903818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-c893a981-8947-4ba3-bd30-abb9bb3d3076 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6d78a27972e44a3790dc30c38b0759a8.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6d78a27972e44a3790dc30c38b0759a8. Aug 10 16:55:47.903818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:47.903818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 16:55:47.903818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 16:55:47.903818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 16:55:47.903818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 16:55:47.903818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 16:55:47.903818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 16:55:47.903818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 16:55:47.903818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application conn.query(q) Aug 10 16:55:47.903818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 16:55:47.903818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 16:55:47.903818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 16:55:47.904721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application result.read() Aug 10 16:55:47.904721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 16:55:47.904721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 16:55:47.904721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 16:55:47.904721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 16:55:47.904721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 16:55:47.904721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 16:55:47.904721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 16:55:47.904721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 16:55:47.904721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 16:55:47.904721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:47.904721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 16:55:47.904721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:47.904721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:47.904721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 16:55:47.904721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 16:55:47.904721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 10 16:55:47.904721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 10 16:55:47.905693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 16:55:47.905693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application next(self.gen) Aug 10 16:55:47.905693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 16:55:47.905693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application yield resource Aug 10 16:55:47.905693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 16:55:47.905693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application next(self.gen) Aug 10 16:55:47.905693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 16:55:47.905693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 16:55:47.905693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 16:55:47.905693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 16:55:47.905693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 16:55:47.905693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise self.value Aug 10 16:55:47.905693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 16:55:47.905693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 16:55:47.905693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 16:55:47.905693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application session.commit() Aug 10 16:55:47.905693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 16:55:47.906651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 16:55:47.906651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 16:55:47.906651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 16:55:47.906651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 16:55:47.906651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self.session.flush() Aug 10 16:55:47.906651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 16:55:47.906651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 16:55:47.906651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 16:55:47.906651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 16:55:47.906651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 16:55:47.906651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application compat.raise_( Aug 10 16:55:47.906651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 16:55:47.906651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception Aug 10 16:55:47.906651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 16:55:47.906651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 16:55:47.906651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 16:55:47.906651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 16:55:47.907592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 16:55:47.907592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 16:55:47.907592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 16:55:47.907592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 16:55:47.907592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 10 16:55:47.907592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 10 16:55:47.907592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 16:55:47.907592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 16:55:47.907592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 16:55:47.907592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 16:55:47.907592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 16:55:47.907592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 16:55:47.907592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 16:55:47.907592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 16:55:47.907592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 16:55:47.908846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 16:55:47.908846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 16:55:47.908846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception Aug 10 16:55:47.908846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 16:55:47.908846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 16:55:47.908846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 16:55:47.908846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 16:55:47.908846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 16:55:47.908846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 16:55:47.908846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 16:55:47.908846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application conn.query(q) Aug 10 16:55:47.908846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 16:55:47.908846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 16:55:47.908846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 16:55:47.908846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application result.read() Aug 10 16:55:47.908846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 16:55:47.908846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 16:55:47.909743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 16:55:47.909743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 16:55:47.909743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 16:55:47.909743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 16:55:47.909743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 16:55:47.909743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 16:55:47.909743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 16:55:47.909743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 10 16:55:47.909743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [parameters: {'id': '6d78a27972e44a3790dc30c38b0759a8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 10 16:55:47.909743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 16:55:47.909743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:47.909743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:55:47.909743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:47.909743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:47.909743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 10 16:55:47.914740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 10 16:55:47.914740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 16:55:47.914740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 16:55:47.914740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 10 16:55:47.914740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:47.914740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:55:47.914740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:47.914740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:47.914740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:47.914740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:47.914740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:47.914740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:47.914740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:47.914740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:47.914740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:47.914740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:47.914740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:47.915840 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:47.915840 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 10 16:55:47.915840 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 10 16:55:47.915840 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:47.915840 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:47.915840 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 10 16:55:47.915840 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 10 16:55:47.915840 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6d78a27972e44a3790dc30c38b0759a8. Aug 10 16:55:47.915840 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:47.915840 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 465/938] 10.222.0.45 () {60 vars in 1140 bytes} [Wed Aug 10 16:55:47 2022] POST /identity/v3/projects => generated 262 bytes in 255 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:55:47.918942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893a981-8947-4ba3-bd30-abb9bb3d3076 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:47.952046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:47.969972 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:48.009000 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e854fe0-b22c-469a-8ef8-70bfe6678bc5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:48.028104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893a981-8947-4ba3-bd30-abb9bb3d3076 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:48.049052 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb69b2-71b1-4017-996e-90a4cec977b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:48.054436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb69b2-71b1-4017-996e-90a4cec977b2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:48.054436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb69b2-71b1-4017-996e-90a4cec977b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:48.054436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb69b2-71b1-4017-996e-90a4cec977b2 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:48.054436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb69b2-71b1-4017-996e-90a4cec977b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:48.054436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb69b2-71b1-4017-996e-90a4cec977b2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:48.065921 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 474/939] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 10 16:55:47 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:55:48.079081 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60289a29-5d1b-4e4d-b73f-0df5b8061dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:48.079752 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60289a29-5d1b-4e4d-b73f-0df5b8061dff None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:48.080084 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60289a29-5d1b-4e4d-b73f-0df5b8061dff None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:48.080369 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60289a29-5d1b-4e4d-b73f-0df5b8061dff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:48.088838 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60289a29-5d1b-4e4d-b73f-0df5b8061dff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:48.090832 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60289a29-5d1b-4e4d-b73f-0df5b8061dff None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:48.104664 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-60289a29-5d1b-4e4d-b73f-0df5b8061dff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:48.104664 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:48.104664 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:48.104664 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:48.104664 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:48.104664 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:48.104664 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:48.104664 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:48.104664 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:48.104664 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:48.104664 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:48.104664 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:48.104664 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:48.104664 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:48.104664 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:48.105801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:48.105801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:48.105801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:48.105801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:48.105801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:48.105801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:48.105801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:48.105801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:48.105801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:48.105801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:48.105801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:48.105801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:48.105801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:48.105801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:48.105801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:48.105801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:48.105801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:48.107412 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:48.107412 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:48.107412 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:48.107412 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:48.107412 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:48.107412 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:48.107412 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:48.107412 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:48.107412 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 466/940] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:48.112258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb69b2-71b1-4017-996e-90a4cec977b2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:48.235549 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb69b2-71b1-4017-996e-90a4cec977b2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:48.253685 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:48.275513 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e27b8-f8ea-4db6-a1fc-6af2d1f89371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:48.276389 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e27b8-f8ea-4db6-a1fc-6af2d1f89371 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:48.276677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e27b8-f8ea-4db6-a1fc-6af2d1f89371 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:48.276952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e27b8-f8ea-4db6-a1fc-6af2d1f89371 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:48.277620 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6e27b8-f8ea-4db6-a1fc-6af2d1f89371 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:48.277922 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6e27b8-f8ea-4db6-a1fc-6af2d1f89371 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:48.279391 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6e27b8-f8ea-4db6-a1fc-6af2d1f89371 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:48.298498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 475/941] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 10 16:55:48 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:48.303968 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b77252-66e9-46bf-a2f2-b35e2eba2ced None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:48.304486 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b77252-66e9-46bf-a2f2-b35e2eba2ced None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:48.304486 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b77252-66e9-46bf-a2f2-b35e2eba2ced None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:48.310146 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 467/942] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:48.315960 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfdda3-f3ee-4325-9666-4c316c59c354 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:48.316291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfdda3-f3ee-4325-9666-4c316c59c354 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:48.316516 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfdda3-f3ee-4325-9666-4c316c59c354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:48.336168 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-fddfdda3-f3ee-4325-9666-4c316c59c354 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:48.337374 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fddfdda3-f3ee-4325-9666-4c316c59c354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:48.415333 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 476/943] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:55:48.439375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b77252-66e9-46bf-a2f2-b35e2eba2ced None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:48.553461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b77252-66e9-46bf-a2f2-b35e2eba2ced None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:48.556865 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:48.591704 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955a7c3-54c0-4853-abe5-b5ecaebdd72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:48.592409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955a7c3-54c0-4853-abe5-b5ecaebdd72a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:48.592685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955a7c3-54c0-4853-abe5-b5ecaebdd72a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:48.592953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955a7c3-54c0-4853-abe5-b5ecaebdd72a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:48.608033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955a7c3-54c0-4853-abe5-b5ecaebdd72a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:48.612818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-5955a7c3-54c0-4853-abe5-b5ecaebdd72a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 16:55:48.612818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:48.612818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:48.612818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:48.612818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:48.612818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:48.612818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:48.612818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:48.612818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:48.612818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:48.612818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:48.612818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:48.612818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:48.612818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:48.612818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:48.612818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:48.613785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:48.613785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:48.613785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:48.613785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:48.613785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:48.613785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:48.613785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:48.613785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:48.613785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:48.613785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:48.613785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:48.613785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:48.613785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:48.613785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:48.613785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:48.613785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:48.615088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:48.615088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:48.615088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:48.615088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:48.615088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:48.615088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:48.615088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:48.615088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:48.615088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:48.615088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:48.615088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 16:55:48.615088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:48.617569 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 468/944] 10.222.0.45 () {56 vars in 1120 bytes} [Wed Aug 10 16:55:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:48.621438 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfdda3-f3ee-4325-9666-4c316c59c354 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:48.731052 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfdda3-f3ee-4325-9666-4c316c59c354 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:48.733594 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:48.767824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dcf82-d367-4ddd-8605-d327bf27b93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:48.769000 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dcf82-d367-4ddd-8605-d327bf27b93e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:48.769470 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dcf82-d367-4ddd-8605-d327bf27b93e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:48.769927 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dcf82-d367-4ddd-8605-d327bf27b93e None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:48.770804 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1dcf82-d367-4ddd-8605-d327bf27b93e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:48.771313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1dcf82-d367-4ddd-8605-d327bf27b93e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:48.773212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1dcf82-d367-4ddd-8605-d327bf27b93e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:48.783755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 477/945] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Aug 10 16:55:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:48.789720 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955a7c3-54c0-4853-abe5-b5ecaebdd72a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:48.901855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955a7c3-54c0-4853-abe5-b5ecaebdd72a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:48.938847 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264568eb-89d3-4ac5-b814-01fb018e4ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:48.939473 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264568eb-89d3-4ac5-b814-01fb018e4ef2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:48.939750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264568eb-89d3-4ac5-b814-01fb018e4ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:48.940019 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264568eb-89d3-4ac5-b814-01fb018e4ef2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:48.952125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264568eb-89d3-4ac5-b814-01fb018e4ef2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:48.953596 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264568eb-89d3-4ac5-b814-01fb018e4ef2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:48.968750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-264568eb-89d3-4ac5-b814-01fb018e4ef2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:48.968750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:48.968750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:48.968750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:48.968750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:48.968750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:48.968750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:48.968750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:48.968750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:48.968750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:48.968750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:48.968750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:48.968750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:48.968750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:48.968750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:48.968750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:48.969778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:48.969778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:48.969778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:48.969778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:48.969778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:48.969778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:48.969778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:48.969778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:48.969778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:48.969778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:48.969778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:48.969778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:48.969778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:48.969778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:48.969778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:48.969778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:48.969778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:48.971141 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:48.971141 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:48.971141 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:48.971141 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 469/946] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:48.975910 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dcf82-d367-4ddd-8605-d327bf27b93e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:49.014072 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62fa97-6fb8-40fa-8838-f457939f084e None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:49.014344 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62fa97-6fb8-40fa-8838-f457939f084e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:49.014689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62fa97-6fb8-40fa-8838-f457939f084e None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:49.016305 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 470/947] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:55:49.021283 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8adc7a-6135-4277-aeed-25017504bfd5 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:49.021634 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8adc7a-6135-4277-aeed-25017504bfd5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:49.021838 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8adc7a-6135-4277-aeed-25017504bfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:49.041110 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ec8adc7a-6135-4277-aeed-25017504bfd5 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:49.043331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ec8adc7a-6135-4277-aeed-25017504bfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:49.054769 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:49.089914 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dcf82-d367-4ddd-8605-d327bf27b93e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:49.128354 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd290d0-81fb-4792-a417-758b57f023b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:49.129172 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd290d0-81fb-4792-a417-758b57f023b8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:49.129451 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd290d0-81fb-4792-a417-758b57f023b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:49.129722 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd290d0-81fb-4792-a417-758b57f023b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:49.130426 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd290d0-81fb-4792-a417-758b57f023b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:49.130677 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 471/948] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:55:49.130782 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd290d0-81fb-4792-a417-758b57f023b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:49.132152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd290d0-81fb-4792-a417-758b57f023b8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:49.147225 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 478/949] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:49.152212 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8adc7a-6135-4277-aeed-25017504bfd5 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:49.153369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd290d0-81fb-4792-a417-758b57f023b8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:49.185404 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:49.270858 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd290d0-81fb-4792-a417-758b57f023b8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:49.277953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8adc7a-6135-4277-aeed-25017504bfd5 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:49.309378 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ca01e-7896-44b7-b001-7a301a55a952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:49.310119 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ca01e-7896-44b7-b001-7a301a55a952 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:49.310523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ca01e-7896-44b7-b001-7a301a55a952 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:49.310792 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ca01e-7896-44b7-b001-7a301a55a952 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:49.313641 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936577f-365c-4c6a-b24d-d5f58e0cefe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:49.314314 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936577f-365c-4c6a-b24d-d5f58e0cefe3 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:49.314685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936577f-365c-4c6a-b24d-d5f58e0cefe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:49.315021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936577f-365c-4c6a-b24d-d5f58e0cefe3 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:49.315735 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0936577f-365c-4c6a-b24d-d5f58e0cefe3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:49.317176 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0936577f-365c-4c6a-b24d-d5f58e0cefe3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:49.318844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ca01e-7896-44b7-b001-7a301a55a952 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:49.320343 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ca01e-7896-44b7-b001-7a301a55a952 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:49.333174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d36ca01e-7896-44b7-b001-7a301a55a952 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 16:55:49.333174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:49.333174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:49.333174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:49.333174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:49.333174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:49.333174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:49.333174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:49.333174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:49.333174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:49.333174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:49.333174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:49.333174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:49.333174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:49.333174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:49.333174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:49.337016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:49.337016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:49.337016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:49.337016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:49.337016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:49.337016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:49.337016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:49.337016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:49.337016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:49.337016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:49.337016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:49.337016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:49.337016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:49.337016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:49.337016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:49.337016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:49.337016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:49.337016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:49.338003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:49.338003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:49.338003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:49.338003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:49.338003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:49.338003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:49.338003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:49.338003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:49.338003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 16:55:49.338003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:49.338003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 479/950] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 10 16:55:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:49.340110 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ca01e-7896-44b7-b001-7a301a55a952 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:49.406946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 472/951] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 10 16:55:49 2022] POST /identity/v3/services => generated 242 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:49.444949 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:49.462988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ca01e-7896-44b7-b001-7a301a55a952 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:49.511083 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be191a74-7915-493e-9c61-c421e30606b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:49.511692 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be191a74-7915-493e-9c61-c421e30606b1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:49.511969 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be191a74-7915-493e-9c61-c421e30606b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:49.512239 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be191a74-7915-493e-9c61-c421e30606b1 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:49.513052 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be191a74-7915-493e-9c61-c421e30606b1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:49.513359 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be191a74-7915-493e-9c61-c421e30606b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:49.514857 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be191a74-7915-493e-9c61-c421e30606b1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:49.536429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 480/952] 10.222.0.45 () {56 vars in 1149 bytes} [Wed Aug 10 16:55:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:49.543030 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936577f-365c-4c6a-b24d-d5f58e0cefe3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:49.552449 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:49.663660 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936577f-365c-4c6a-b24d-d5f58e0cefe3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:49.702383 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24f975-fc6c-4b3f-893b-462044515b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:49.703415 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24f975-fc6c-4b3f-893b-462044515b13 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:49.703882 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24f975-fc6c-4b3f-893b-462044515b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:49.704327 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24f975-fc6c-4b3f-893b-462044515b13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:49.716133 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24f975-fc6c-4b3f-893b-462044515b13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:49.717984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24f975-fc6c-4b3f-893b-462044515b13 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:49.734064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-cd24f975-fc6c-4b3f-893b-462044515b13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:49.734064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:49.734064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:49.734064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:49.734064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:49.734064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:49.734064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:49.734064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:49.734064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:49.734064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:49.734064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:49.734064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:49.734064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:49.734064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:49.734064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:49.734064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:49.735096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:49.735096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:49.735096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:49.735096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:49.735096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:49.735096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:49.735096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:49.735096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:49.735096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:49.735096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:49.735096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:49.735096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:49.735096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:49.735096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:49.735096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:49.735096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:49.735096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:49.736206 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:49.736206 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:49.736206 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:49.738267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 473/953] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:49.744091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be191a74-7915-493e-9c61-c421e30606b1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:49.851819 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be191a74-7915-493e-9c61-c421e30606b1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:49.889363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c26615-155e-43c1-9288-c95075f20f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:49.890221 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c26615-155e-43c1-9288-c95075f20f9b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:49.890697 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c26615-155e-43c1-9288-c95075f20f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:49.891175 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c26615-155e-43c1-9288-c95075f20f9b None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:49.892018 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c26615-155e-43c1-9288-c95075f20f9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:49.892481 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c26615-155e-43c1-9288-c95075f20f9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:49.894002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c26615-155e-43c1-9288-c95075f20f9b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:49.906704 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:49.909968 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 481/954] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:49.912471 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24f975-fc6c-4b3f-893b-462044515b13 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:49.993135 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:50.058383 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24f975-fc6c-4b3f-893b-462044515b13 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:50.133089 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115af5c2-8bee-4801-b075-2c47f4314b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:50.133807 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115af5c2-8bee-4801-b075-2c47f4314b42 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:50.134090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115af5c2-8bee-4801-b075-2c47f4314b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:50.134373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115af5c2-8bee-4801-b075-2c47f4314b42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:50.147405 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115af5c2-8bee-4801-b075-2c47f4314b42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:50.149120 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115af5c2-8bee-4801-b075-2c47f4314b42 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:50.165961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-115af5c2-8bee-4801-b075-2c47f4314b42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:50.165961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:50.165961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:50.165961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:50.165961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:50.165961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:50.165961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:50.165961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:50.165961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:50.165961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:50.165961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:50.165961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:50.165961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:50.165961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:50.165961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:50.167688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:50.167688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:50.167688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:50.167688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:50.167688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:50.167688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:50.167688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:50.167688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:50.167688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:50.167688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:50.167688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:50.167688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:50.167688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:50.167688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:50.167688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:50.167688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:50.167688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:50.168825 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:50.168825 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:50.168825 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:50.168825 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:50.168825 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:50.168825 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:50.168825 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:50.168825 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:50.178608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 474/955] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:50.191507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c26615-155e-43c1-9288-c95075f20f9b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:50.297645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3dfc9-7ee3-41b6-a1a2-fa6e6e88baf9 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:50.297821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3dfc9-7ee3-41b6-a1a2-fa6e6e88baf9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:50.298118 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3dfc9-7ee3-41b6-a1a2-fa6e6e88baf9 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:50.302600 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 475/956] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:55:50.305694 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd4a92-d5c3-4025-a8ec-5642fd60a322 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:50.305954 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd4a92-d5c3-4025-a8ec-5642fd60a322 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:50.306219 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd4a92-d5c3-4025-a8ec-5642fd60a322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:50.336079 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-25dd4a92-d5c3-4025-a8ec-5642fd60a322 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:50.337266 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-25dd4a92-d5c3-4025-a8ec-5642fd60a322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:50.386328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c26615-155e-43c1-9288-c95075f20f9b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:50.430001 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c28cd-bfc2-40be-bebb-9d77eb218ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:50.430663 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c28cd-bfc2-40be-bebb-9d77eb218ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:50.431023 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c28cd-bfc2-40be-bebb-9d77eb218ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:50.431297 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c28cd-bfc2-40be-bebb-9d77eb218ae1 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:50.432034 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c28cd-bfc2-40be-bebb-9d77eb218ae1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:50.432909 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c28cd-bfc2-40be-bebb-9d77eb218ae1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:50.434558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c28cd-bfc2-40be-bebb-9d77eb218ae1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:50.448557 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 482/957] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:50.454101 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c28cd-bfc2-40be-bebb-9d77eb218ae1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:50.504539 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 476/958] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:50 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:55:50.527391 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd4a92-d5c3-4025-a8ec-5642fd60a322 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:50.540726 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:50.579898 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:50.598247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c28cd-bfc2-40be-bebb-9d77eb218ae1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:50.645099 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd4a92-d5c3-4025-a8ec-5642fd60a322 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:50.645336 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a71c2-6c8f-4baf-b334-ecb75142fd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:50.645953 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a71c2-6c8f-4baf-b334-ecb75142fd0b None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:50.646229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a71c2-6c8f-4baf-b334-ecb75142fd0b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:50.646570 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a71c2-6c8f-4baf-b334-ecb75142fd0b None admin] PATH_INFO: `/v3/projects/d11f33af53374dffa61132642a7770f4/users/4f9e1767ec90435f80248a2930081265/roles/c37c16d90ff44d80991a2e4bdb50432c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:50.649770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a71c2-6c8f-4baf-b334-ecb75142fd0b None admin] RBAC: Authorizing `identity:create_grant(project_id=d11f33af53374dffa61132642a7770f4, user_id=4f9e1767ec90435f80248a2930081265, role_id=c37c16d90ff44d80991a2e4bdb50432c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:50.652103 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a71c2-6c8f-4baf-b334-ecb75142fd0b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:50.681052 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a58081-55b6-482a-96cf-a5f0c595e109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:50.681677 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a58081-55b6-482a-96cf-a5f0c595e109 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:50.681973 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a58081-55b6-482a-96cf-a5f0c595e109 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:50.682778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a58081-55b6-482a-96cf-a5f0c595e109 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:50.683470 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a58081-55b6-482a-96cf-a5f0c595e109 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:50.684835 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a58081-55b6-482a-96cf-a5f0c595e109 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:50.694389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-b3a58081-55b6-482a-96cf-a5f0c595e109 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:55:50.694389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:50.694389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:55:50.694389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:55:50.694389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:55:50.694389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:55:50.694389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:55:50.694389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:50.694389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:55:50.694389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:50.694389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:55:50.694389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:50.694389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:50.694389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:50.694389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:50.694389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:50.694389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:50.699310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:50.699310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:50.699310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:50.699310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:50.699310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:50.699310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:50.699310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:55:50.699310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:55:50.699310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:55:50.699310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:55:50.699310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:50.699310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:50.699310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:50.699310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:50.699310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:50.699310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:50.700198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:50.700198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:50.700198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:50.700198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:50.700198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:50.700198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:50.700198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:50.700198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:50.700198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:50.700198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:50.700198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:55:50.700198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:50.700198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:55:50.700198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:50.700198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 477/959] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 10 16:55:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:50.701762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85950bc-0be8-4b2b-ac15-f58feac4a5e7 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:50.701762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85950bc-0be8-4b2b-ac15-f58feac4a5e7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:50.701762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85950bc-0be8-4b2b-ac15-f58feac4a5e7 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:50.701762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 478/960] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:50.705414 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdc8bb-df3c-4b45-aa6e-265438860bf9 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:50.705686 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdc8bb-df3c-4b45-aa6e-265438860bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:50.705950 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdc8bb-df3c-4b45-aa6e-265438860bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:50.723852 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-cdcdc8bb-df3c-4b45-aa6e-265438860bf9 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:50.727214 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cdcdc8bb-df3c-4b45-aa6e-265438860bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:50.762883 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 483/961] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 10 16:55:50 2022] PUT /identity/v3/projects/d11f33af53374dffa61132642a7770f4/users/4f9e1767ec90435f80248a2930081265/roles/c37c16d90ff44d80991a2e4bdb50432c => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:55:50.764079 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a71c2-6c8f-4baf-b334-ecb75142fd0b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:50.812269 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 479/962] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:50 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:55:50.840130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcdc8bb-df3c-4b45-aa6e-265438860bf9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:50.890803 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a71c2-6c8f-4baf-b334-ecb75142fd0b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:50.893373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:50.915990 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:50.936055 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd151fe-b766-4e10-82f4-80cf633ae273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:50.936771 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd151fe-b766-4e10-82f4-80cf633ae273 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:50.937111 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd151fe-b766-4e10-82f4-80cf633ae273 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:50.938233 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd151fe-b766-4e10-82f4-80cf633ae273 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:50.938990 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd151fe-b766-4e10-82f4-80cf633ae273 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:50.939355 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd151fe-b766-4e10-82f4-80cf633ae273 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:50.941805 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd151fe-b766-4e10-82f4-80cf633ae273 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:50.949788 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 484/963] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 10 16:55:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:50.960441 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd151fe-b766-4e10-82f4-80cf633ae273 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:50.979669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcdc8bb-df3c-4b45-aa6e-265438860bf9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:51.017482 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93e064-f28a-451b-b3bd-68f5e4763ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:51.018494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93e064-f28a-451b-b3bd-68f5e4763ca1 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:51.019000 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93e064-f28a-451b-b3bd-68f5e4763ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:51.019323 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93e064-f28a-451b-b3bd-68f5e4763ca1 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:51.019973 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c93e064-f28a-451b-b3bd-68f5e4763ca1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:51.021435 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c93e064-f28a-451b-b3bd-68f5e4763ca1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:51.078685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.sql.core [None req-4c93e064-f28a-451b-b3bd-68f5e4763ca1 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 16:55:51.078685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 10 16:55:51.078685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [parameters: {'user_id': '5e97bab96b84486683bf934415efdabf', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 10 16:55:51.078685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81612) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 16:55:51.085106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-4c93e064-f28a-451b-b3bd-68f5e4763ca1 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 10 16:55:51.085106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:51.085106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 16:55:51.085106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 16:55:51.085106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 16:55:51.085106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 16:55:51.085106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 16:55:51.085106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 16:55:51.085106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 16:55:51.085106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application conn.query(q) Aug 10 16:55:51.085106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 16:55:51.085106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 16:55:51.085106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 16:55:51.085106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application result.read() Aug 10 16:55:51.085106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 16:55:51.086043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 16:55:51.086043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 16:55:51.086043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 16:55:51.086043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 16:55:51.086043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 16:55:51.086043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 16:55:51.086043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 16:55:51.086043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 16:55:51.086043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:51.086043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 16:55:51.086043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:51.086043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:51.086043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 16:55:51.086043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 16:55:51.086043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 10 16:55:51.086043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 10 16:55:51.086043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 16:55:51.086043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application next(self.gen) Aug 10 16:55:51.087070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 16:55:51.087070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application yield resource Aug 10 16:55:51.087070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 16:55:51.087070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application next(self.gen) Aug 10 16:55:51.087070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 16:55:51.087070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 16:55:51.087070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 16:55:51.087070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 16:55:51.087070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 16:55:51.087070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise self.value Aug 10 16:55:51.087070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 16:55:51.087070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 16:55:51.087070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 16:55:51.087070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application session.commit() Aug 10 16:55:51.087070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 16:55:51.087070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 16:55:51.087070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 16:55:51.087980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 16:55:51.087980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 16:55:51.087980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self.session.flush() Aug 10 16:55:51.087980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 16:55:51.087980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 16:55:51.087980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 16:55:51.087980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 16:55:51.087980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 16:55:51.087980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application compat.raise_( Aug 10 16:55:51.087980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 16:55:51.087980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception Aug 10 16:55:51.087980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 16:55:51.087980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 16:55:51.087980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 16:55:51.087980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 16:55:51.087980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 16:55:51.087980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 16:55:51.089402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 16:55:51.089402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 16:55:51.089402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 10 16:55:51.089402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 10 16:55:51.089402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 16:55:51.089402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 16:55:51.089402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 16:55:51.089402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 16:55:51.089402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 16:55:51.089402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 16:55:51.089402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 16:55:51.089402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 16:55:51.089402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 16:55:51.089402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 16:55:51.089402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 16:55:51.089402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception Aug 10 16:55:51.090321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 16:55:51.090321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 16:55:51.090321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 16:55:51.090321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 16:55:51.090321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 16:55:51.090321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 16:55:51.090321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 16:55:51.090321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application conn.query(q) Aug 10 16:55:51.090321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 16:55:51.090321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 16:55:51.090321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 16:55:51.090321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application result.read() Aug 10 16:55:51.090321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 16:55:51.090321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 16:55:51.090321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 16:55:51.090321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 16:55:51.093275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 16:55:51.093275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 16:55:51.093275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 16:55:51.093275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 16:55:51.093275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 16:55:51.093275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 10 16:55:51.093275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [parameters: {'user_id': '5e97bab96b84486683bf934415efdabf', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 10 16:55:51.093275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 16:55:51.093275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:51.093275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:55:51.093275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:51.093275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:51.093275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:51.093275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:51.093275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:51.108165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:51.108165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:51.108165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:51.108165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:51.108165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:51.108165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:51.108165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:51.108165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 10 16:55:51.108165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 10 16:55:51.108165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:51.108165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:51.108165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:51.108165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:51.108165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:51.108165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:51.108165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 10 16:55:51.108165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 10 16:55:51.109333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 10 16:55:51.109333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 10 16:55:51.109333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 16:55:51.109333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 16:55:51.109333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 10 16:55:51.109333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:51.109333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 480/964] 10.222.0.45 () {60 vars in 1131 bytes} [Wed Aug 10 16:55:50 2022] POST /identity/v3/users => generated 160 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:51.109333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93e064-f28a-451b-b3bd-68f5e4763ca1 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:51.109910 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd151fe-b766-4e10-82f4-80cf633ae273 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:51.151281 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a96da-3866-4309-a94b-8d47ff54a875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:51.151901 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a96da-3866-4309-a94b-8d47ff54a875 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:51.152187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a96da-3866-4309-a94b-8d47ff54a875 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:51.152463 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a96da-3866-4309-a94b-8d47ff54a875 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:51.153367 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387a96da-3866-4309-a94b-8d47ff54a875 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:51.153670 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387a96da-3866-4309-a94b-8d47ff54a875 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:51.155505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387a96da-3866-4309-a94b-8d47ff54a875 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:51.163072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 485/965] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 10 16:55:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:51.169910 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a96da-3866-4309-a94b-8d47ff54a875 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:51.215558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93e064-f28a-451b-b3bd-68f5e4763ca1 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:51.218308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:51.238008 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:51.254764 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d5c6e-6a32-4148-99f9-27d8100752e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:51.255768 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d5c6e-6a32-4148-99f9-27d8100752e4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:51.256254 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d5c6e-6a32-4148-99f9-27d8100752e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:51.256730 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d5c6e-6a32-4148-99f9-27d8100752e4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:51.266681 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d5c6e-6a32-4148-99f9-27d8100752e4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:51.273461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d5c6e-6a32-4148-99f9-27d8100752e4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:51.281942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-c24d5c6e-6a32-4148-99f9-27d8100752e4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 16:55:51.281942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:51.281942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:51.281942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:51.281942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:51.281942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:51.281942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:51.281942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:51.281942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:51.281942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:51.281942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:51.281942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:51.281942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:51.281942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:51.281942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:51.281942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:51.282965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:51.282965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:51.282965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:51.282965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:51.282965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:51.282965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:51.282965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:51.282965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:51.282965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:51.282965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:51.282965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:51.282965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:51.282965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:51.282965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:51.282965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:51.282965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:51.282965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:51.282965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:51.283936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:51.283936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:51.283936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:51.283936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:51.283936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:51.283936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:51.283936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:51.283936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:51.283936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 16:55:51.283936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:51.291915 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 481/966] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:55:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:51.293581 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d5c6e-6a32-4148-99f9-27d8100752e4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:51.301079 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a96da-3866-4309-a94b-8d47ff54a875 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:51.348291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd73134-7d46-4199-a171-922b19c9dcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:51.349119 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd73134-7d46-4199-a171-922b19c9dcd8 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:51.349406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd73134-7d46-4199-a171-922b19c9dcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:51.349682 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd73134-7d46-4199-a171-922b19c9dcd8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:51.350367 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd73134-7d46-4199-a171-922b19c9dcd8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:51.351761 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd73134-7d46-4199-a171-922b19c9dcd8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:51.403848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d5c6e-6a32-4148-99f9-27d8100752e4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:51.439396 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3151a84-28bb-4acb-b81f-54d5d504f397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:51.440153 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3151a84-28bb-4acb-b81f-54d5d504f397 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:51.440522 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3151a84-28bb-4acb-b81f-54d5d504f397 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:51.440882 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3151a84-28bb-4acb-b81f-54d5d504f397 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:51.441698 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3151a84-28bb-4acb-b81f-54d5d504f397 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:51.442106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3151a84-28bb-4acb-b81f-54d5d504f397 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:51.443687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3151a84-28bb-4acb-b81f-54d5d504f397 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:51.451891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 486/967] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 16:55:51 2022] POST /identity/v3/endpoints => generated 349 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:51.502546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 482/968] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 10 16:55:51 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:51.700411 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954c4dc-bdc1-4613-a293-21b4edada408 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:51.700851 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954c4dc-bdc1-4613-a293-21b4edada408 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:51.701224 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954c4dc-bdc1-4613-a293-21b4edada408 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:51.702876 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 487/969] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:51.708672 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cd6e5-0f72-49d0-851f-c5c7475c9e8c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:51.709074 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cd6e5-0f72-49d0-851f-c5c7475c9e8c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:51.709463 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cd6e5-0f72-49d0-851f-c5c7475c9e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:51.730478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e46cd6e5-0f72-49d0-851f-c5c7475c9e8c None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:51.732183 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e46cd6e5-0f72-49d0-851f-c5c7475c9e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:51.751073 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:51.822161 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 483/970] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:51 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:55:51.844080 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954c4dc-bdc1-4613-a293-21b4edada408 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:51.954245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954c4dc-bdc1-4613-a293-21b4edada408 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:51.957182 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:51.991612 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46acd601-3d2e-415e-9043-dfa9cc4d2970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:51.992303 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46acd601-3d2e-415e-9043-dfa9cc4d2970 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:51.992585 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46acd601-3d2e-415e-9043-dfa9cc4d2970 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:51.992852 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46acd601-3d2e-415e-9043-dfa9cc4d2970 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:51.993434 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46acd601-3d2e-415e-9043-dfa9cc4d2970 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:51.994906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46acd601-3d2e-415e-9043-dfa9cc4d2970 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:52.004523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-46acd601-3d2e-415e-9043-dfa9cc4d2970 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 16:55:52.004523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:52.004523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:55:52.004523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:55:52.004523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:55:52.004523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:55:52.004523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:55:52.004523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:52.004523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 16:55:52.004523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:52.004523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:55:52.004523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:52.004523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:52.004523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:52.004523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:52.004523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:52.004523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:52.005776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:52.005776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:52.005776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:52.005776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:52.005776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:52.005776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:52.005776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:55:52.005776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:55:52.005776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:55:52.005776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:55:52.005776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:52.005776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:52.005776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:52.005776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:52.005776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:52.005776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:52.006880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:52.006880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:52.006880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:52.006880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:52.006880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:52.006880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:52.006880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:52.006880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:52.006880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:52.006880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:52.006880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:55:52.006880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:52.006880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 16:55:52.006880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:52.014403 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 488/971] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:52.016307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cd6e5-0f72-49d0-851f-c5c7475c9e8c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:52.124712 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cd6e5-0f72-49d0-851f-c5c7475c9e8c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:52.160844 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5301ecc1-0ea8-462d-9973-7f96a3f94a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:52.161444 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301ecc1-0ea8-462d-9973-7f96a3f94a5c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:52.161714 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301ecc1-0ea8-462d-9973-7f96a3f94a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:52.161976 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301ecc1-0ea8-462d-9973-7f96a3f94a5c None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:52.162641 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5301ecc1-0ea8-462d-9973-7f96a3f94a5c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:52.162946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5301ecc1-0ea8-462d-9973-7f96a3f94a5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:52.164354 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5301ecc1-0ea8-462d-9973-7f96a3f94a5c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:52.175383 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 484/972] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 10 16:55:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:52.176719 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46acd601-3d2e-415e-9043-dfa9cc4d2970 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:52.283473 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46acd601-3d2e-415e-9043-dfa9cc4d2970 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:52.318162 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd0d6d-0a7c-4f73-ad83-19193e3e20ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:52.319024 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd0d6d-0a7c-4f73-ad83-19193e3e20ca None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:52.319401 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd0d6d-0a7c-4f73-ad83-19193e3e20ca None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:52.319775 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd0d6d-0a7c-4f73-ad83-19193e3e20ca None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:52.320510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd0d6d-0a7c-4f73-ad83-19193e3e20ca None admin] RBAC: Adding query filter params (type=compute) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:52.320916 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd0d6d-0a7c-4f73-ad83-19193e3e20ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:52.322413 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd0d6d-0a7c-4f73-ad83-19193e3e20ca None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:52.333090 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 489/973] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 10 16:55:52 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:52.334437 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5301ecc1-0ea8-462d-9973-7f96a3f94a5c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:52.351464 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:52.443285 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5301ecc1-0ea8-462d-9973-7f96a3f94a5c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:52.482774 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014cb90d-4383-4c74-b14b-f8878b48e40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:52.483377 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cb90d-4383-4c74-b14b-f8878b48e40b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:52.483647 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cb90d-4383-4c74-b14b-f8878b48e40b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:52.483911 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cb90d-4383-4c74-b14b-f8878b48e40b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:52.484599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014cb90d-4383-4c74-b14b-f8878b48e40b None admin] RBAC: Adding query filter params (interface=public, service_id=b170ec51097946389bb386c833c3f0b9, region_id=RegionOne) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:52.484898 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014cb90d-4383-4c74-b14b-f8878b48e40b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:52.486288 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014cb90d-4383-4c74-b14b-f8878b48e40b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:52.496586 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 485/974] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Aug 10 16:55:52 2022] GET /identity/v3/endpoints?service_id=b170ec51097946389bb386c833c3f0b9&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:52.497915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd0d6d-0a7c-4f73-ad83-19193e3e20ca None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:52.533737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:52.614701 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd0d6d-0a7c-4f73-ad83-19193e3e20ca None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:52.656374 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3548e336-a6d7-4678-85d5-b0229f740499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:52.656982 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548e336-a6d7-4678-85d5-b0229f740499 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:52.657304 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548e336-a6d7-4678-85d5-b0229f740499 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:52.657579 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548e336-a6d7-4678-85d5-b0229f740499 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:52.658180 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3548e336-a6d7-4678-85d5-b0229f740499 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:52.659574 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3548e336-a6d7-4678-85d5-b0229f740499 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:52.665990 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 490/975] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 10 16:55:52 2022] GET /identity/v3/services => generated 1727 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:55:52.949966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d51cd-e11f-4f4d-b128-1c95490bcef0 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:52.950244 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d51cd-e11f-4f4d-b128-1c95490bcef0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:52.950565 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d51cd-e11f-4f4d-b128-1c95490bcef0 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:52.951951 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 486/976] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:52.957248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee56d0-fd0d-4220-8cd5-a75d3010f938 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:52.957531 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee56d0-fd0d-4220-8cd5-a75d3010f938 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:52.957808 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee56d0-fd0d-4220-8cd5-a75d3010f938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:52.975817 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2eee56d0-fd0d-4220-8cd5-a75d3010f938 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:52.977002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2eee56d0-fd0d-4220-8cd5-a75d3010f938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:53.047262 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 491/977] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:52 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:55:53.071183 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d51cd-e11f-4f4d-b128-1c95490bcef0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:53.152571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:53.176911 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d51cd-e11f-4f4d-b128-1c95490bcef0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:53.212719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a3830-734e-422b-bbd1-6e4d4c89e451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:53.213614 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a3830-734e-422b-bbd1-6e4d4c89e451 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:53.214023 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a3830-734e-422b-bbd1-6e4d4c89e451 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:53.214410 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a3830-734e-422b-bbd1-6e4d4c89e451 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:53.224723 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69a3830-734e-422b-bbd1-6e4d4c89e451 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:53.230195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-f69a3830-734e-422b-bbd1-6e4d4c89e451 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 16:55:53.230195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:53.230195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:53.230195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:53.230195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:53.230195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:53.230195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:53.230195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:53.230195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:53.230195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:53.230195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:53.230195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:53.230195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:53.230195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:53.230195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:53.230195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:53.231376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:53.231376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:53.231376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:53.231376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:53.231376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:53.231376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:53.231376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:53.231376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:53.231376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:53.231376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:53.231376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:53.231376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:53.231376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:53.231376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:53.231376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:53.231376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:53.232495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:53.232495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:53.232495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:53.232495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:53.232495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:53.232495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:53.232495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:53.232495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:53.232495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:53.232495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:53.232495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 16:55:53.232495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:53.233772 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 487/978] 10.222.0.45 () {56 vars in 1120 bytes} [Wed Aug 10 16:55:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:53.239524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee56d0-fd0d-4220-8cd5-a75d3010f938 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:53.266514 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:53.339995 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee56d0-fd0d-4220-8cd5-a75d3010f938 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:53.375825 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e2654-8309-4a0e-a82c-04508c47d3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:53.376458 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e2654-8309-4a0e-a82c-04508c47d3b5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:53.376739 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e2654-8309-4a0e-a82c-04508c47d3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:53.377007 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e2654-8309-4a0e-a82c-04508c47d3b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:53.377669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e2654-8309-4a0e-a82c-04508c47d3b5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:53.377970 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e2654-8309-4a0e-a82c-04508c47d3b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:53.379365 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e2654-8309-4a0e-a82c-04508c47d3b5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:53.392458 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 492/979] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Aug 10 16:55:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:53.393699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a3830-734e-422b-bbd1-6e4d4c89e451 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:53.498726 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a3830-734e-422b-bbd1-6e4d4c89e451 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:53.532645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a27f6c-149e-44a1-9e1a-dbaf84de4758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:53.533265 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a27f6c-149e-44a1-9e1a-dbaf84de4758 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:53.533536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a27f6c-149e-44a1-9e1a-dbaf84de4758 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:53.533800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a27f6c-149e-44a1-9e1a-dbaf84de4758 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:53.536512 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:53.541236 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec3286-c7aa-45ed-9a67-f147c3aef819 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:53.542261 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec3286-c7aa-45ed-9a67-f147c3aef819 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:53.542261 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec3286-c7aa-45ed-9a67-f147c3aef819 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:53.543357 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 493/980] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:55:53.546398 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a27f6c-149e-44a1-9e1a-dbaf84de4758 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:53.547887 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a27f6c-149e-44a1-9e1a-dbaf84de4758 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:53.551772 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29bdc9-9de4-445d-90f9-0fb0c5dcbe68 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:53.551772 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29bdc9-9de4-445d-90f9-0fb0c5dcbe68 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:53.551772 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29bdc9-9de4-445d-90f9-0fb0c5dcbe68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:53.563504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-a0a27f6c-149e-44a1-9e1a-dbaf84de4758 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:53.563504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:53.563504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:53.563504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:53.563504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:53.563504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:53.563504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:53.563504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:53.563504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:53.563504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:53.563504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:53.563504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:53.563504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:53.563504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:53.563504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:53.563504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:53.564558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:53.564558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:53.564558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:53.564558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:53.564558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:53.564558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:53.564558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:53.564558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:53.564558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:53.564558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:53.564558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:53.564558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:53.564558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:53.564558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:53.564558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:53.564558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:53.564558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:53.565478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:53.565478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:53.565478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:53.567142 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-cb29bdc9-9de4-445d-90f9-0fb0c5dcbe68 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:53.568390 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cb29bdc9-9de4-445d-90f9-0fb0c5dcbe68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:53.568941 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 488/981] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:53.570141 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a27f6c-149e-44a1-9e1a-dbaf84de4758 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:53.654915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 494/982] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:53 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:55:53.669065 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29bdc9-9de4-445d-90f9-0fb0c5dcbe68 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:53.687121 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a27f6c-149e-44a1-9e1a-dbaf84de4758 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:53.688856 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:53.722542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d3b5f8-cb51-408c-a458-65df6ff3753c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:53.723148 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3b5f8-cb51-408c-a458-65df6ff3753c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:53.723424 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3b5f8-cb51-408c-a458-65df6ff3753c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:53.723692 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3b5f8-cb51-408c-a458-65df6ff3753c None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:53.724327 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d3b5f8-cb51-408c-a458-65df6ff3753c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:53.724624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d3b5f8-cb51-408c-a458-65df6ff3753c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:53.725975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d3b5f8-cb51-408c-a458-65df6ff3753c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:53.738247 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 489/983] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:53.743859 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d3b5f8-cb51-408c-a458-65df6ff3753c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:53.779115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29bdc9-9de4-445d-90f9-0fb0c5dcbe68 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:53.803648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:53.821010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649bb970-4c16-4af2-9888-3fe689edbc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:53.822037 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bb970-4c16-4af2-9888-3fe689edbc80 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:53.822037 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bb970-4c16-4af2-9888-3fe689edbc80 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:53.822209 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bb970-4c16-4af2-9888-3fe689edbc80 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:53.823517 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649bb970-4c16-4af2-9888-3fe689edbc80 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:53.824902 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649bb970-4c16-4af2-9888-3fe689edbc80 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:53.834002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-649bb970-4c16-4af2-9888-3fe689edbc80 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:55:53.834002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:53.834002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:55:53.834002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:55:53.834002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:55:53.834002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:55:53.834002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:55:53.834002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:53.834002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:55:53.834002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:53.834002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:55:53.834002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:53.834002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:53.834002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:53.834002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:53.834002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:53.834002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:53.835228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:53.835228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:53.835228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:53.835228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:53.835228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:53.835228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:53.835228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:55:53.835228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:55:53.835228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:55:53.835228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:55:53.835228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:53.835228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:53.835228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:53.835228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:53.835228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:53.835228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:53.836074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:53.836074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:53.836074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:53.836074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:53.836074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:53.836074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:53.836074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:53.836074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:53.836074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:53.836074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:53.836074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:55:53.836074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:53.836074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:55:53.836074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:53.836074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 495/984] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 10 16:55:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:55:53.841348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649bb970-4c16-4af2-9888-3fe689edbc80 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:53.863245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d3b5f8-cb51-408c-a458-65df6ff3753c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:53.905051 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3360a-3502-45f7-ac67-a7cbbd5a315b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:53.905703 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3360a-3502-45f7-ac67-a7cbbd5a315b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:53.906068 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3360a-3502-45f7-ac67-a7cbbd5a315b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:53.906270 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3360a-3502-45f7-ac67-a7cbbd5a315b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:53.914013 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3360a-3502-45f7-ac67-a7cbbd5a315b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:53.915486 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3360a-3502-45f7-ac67-a7cbbd5a315b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:53.927104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-f1f3360a-3502-45f7-ac67-a7cbbd5a315b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 16:55:53.927104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:53.927104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:53.927104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:53.927104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:53.927104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:53.927104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:53.927104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:53.927104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:53.927104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:53.927104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:53.927104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:53.927104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:53.927104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:53.927104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:53.927104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:53.928092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:53.928092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:53.928092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:53.928092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:53.928092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:53.928092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:53.928092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:53.928092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:53.928092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:53.928092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:53.928092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:53.928092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:53.928092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:53.928092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:53.928092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:53.928092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:53.928092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:53.928092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:53.929042 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:53.929042 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:53.929042 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:53.929042 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:53.929042 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:53.929042 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:53.929042 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:53.929042 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:53.929042 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 16:55:53.929042 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:53.931313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:53.933618 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 490/985] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 10 16:55:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:55:53.934596 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3360a-3502-45f7-ac67-a7cbbd5a315b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:53.950757 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649bb970-4c16-4af2-9888-3fe689edbc80 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:53.990733 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9385820-a1a8-4438-a781-ea2a911ba6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:53.991337 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9385820-a1a8-4438-a781-ea2a911ba6f7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:53.991615 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9385820-a1a8-4438-a781-ea2a911ba6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:53.991884 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9385820-a1a8-4438-a781-ea2a911ba6f7 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:53.992508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9385820-a1a8-4438-a781-ea2a911ba6f7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:53.992809 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9385820-a1a8-4438-a781-ea2a911ba6f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:53.994168 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9385820-a1a8-4438-a781-ea2a911ba6f7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:54.000288 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 496/986] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 10 16:55:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:54.005750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9385820-a1a8-4438-a781-ea2a911ba6f7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:54.050989 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3360a-3502-45f7-ac67-a7cbbd5a315b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:54.063664 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:54.091822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3695f771-3a6c-45c8-b3dc-4c98ab2d2cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:54.092425 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695f771-3a6c-45c8-b3dc-4c98ab2d2cdb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:54.092701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695f771-3a6c-45c8-b3dc-4c98ab2d2cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:54.092970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695f771-3a6c-45c8-b3dc-4c98ab2d2cdb None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:54.093784 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3695f771-3a6c-45c8-b3dc-4c98ab2d2cdb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:54.094099 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3695f771-3a6c-45c8-b3dc-4c98ab2d2cdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:54.095666 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3695f771-3a6c-45c8-b3dc-4c98ab2d2cdb None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:54.110196 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9385820-a1a8-4438-a781-ea2a911ba6f7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:54.116937 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 491/987] 10.222.0.45 () {56 vars in 1149 bytes} [Wed Aug 10 16:55:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:54.122956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3695f771-3a6c-45c8-b3dc-4c98ab2d2cdb None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:54.144833 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322d8ea-e35d-46ee-8ebf-ae1c363649e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:54.145439 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322d8ea-e35d-46ee-8ebf-ae1c363649e8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:54.145716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322d8ea-e35d-46ee-8ebf-ae1c363649e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:54.145985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322d8ea-e35d-46ee-8ebf-ae1c363649e8 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:54.146645 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d322d8ea-e35d-46ee-8ebf-ae1c363649e8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:54.146945 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d322d8ea-e35d-46ee-8ebf-ae1c363649e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:54.148312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d322d8ea-e35d-46ee-8ebf-ae1c363649e8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:54.154184 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 497/988] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 10 16:55:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:54.160051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322d8ea-e35d-46ee-8ebf-ae1c363649e8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:54.177873 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:54.230448 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3695f771-3a6c-45c8-b3dc-4c98ab2d2cdb None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:54.266288 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b72cd-6c7d-48db-885c-a559b91f98e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:54.266917 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b72cd-6c7d-48db-885c-a559b91f98e8 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:54.267193 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b72cd-6c7d-48db-885c-a559b91f98e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:54.267467 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b72cd-6c7d-48db-885c-a559b91f98e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:54.287013 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322d8ea-e35d-46ee-8ebf-ae1c363649e8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:54.307761 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:54.321964 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cf880-5ca2-401a-b2ee-611913605826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:54.322732 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cf880-5ca2-401a-b2ee-611913605826 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:54.323214 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cf880-5ca2-401a-b2ee-611913605826 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:54.324428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cf880-5ca2-401a-b2ee-611913605826 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:54.324428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cf880-5ca2-401a-b2ee-611913605826 None admin] RBAC: Adding query filter params (interface=admin, service_id=25c79f8c874d4fd59f7058392fe637c8, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:54.324870 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cf880-5ca2-401a-b2ee-611913605826 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:54.326287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cf880-5ca2-401a-b2ee-611913605826 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:54.332088 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 498/989] 10.222.0.45 () {56 vars in 1249 bytes} [Wed Aug 10 16:55:54 2022] GET /identity/v3/endpoints?service_id=25c79f8c874d4fd59f7058392fe637c8&interface=admin®ion_id=RegionOne => generated 193 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:54.337586 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cf880-5ca2-401a-b2ee-611913605826 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:54.349716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b72cd-6c7d-48db-885c-a559b91f98e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:54.351661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b72cd-6c7d-48db-885c-a559b91f98e8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:54.423817 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:54.450679 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cf880-5ca2-401a-b2ee-611913605826 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:54.465583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 492/990] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:55:54 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 16:55:54.471899 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b72cd-6c7d-48db-885c-a559b91f98e8 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:54.486375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c6ffa-0e37-4f21-ab6a-a03b1ce0fe44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:54.487053 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c6ffa-0e37-4f21-ab6a-a03b1ce0fe44 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:54.487329 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c6ffa-0e37-4f21-ab6a-a03b1ce0fe44 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:54.487597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c6ffa-0e37-4f21-ab6a-a03b1ce0fe44 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:54.488197 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c6ffa-0e37-4f21-ab6a-a03b1ce0fe44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:54.489570 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c6ffa-0e37-4f21-ab6a-a03b1ce0fe44 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:54.497674 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 499/991] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 10 16:55:54 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:55:54.548078 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:54.588913 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b72cd-6c7d-48db-885c-a559b91f98e8 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:54.636244 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11959a-224e-46f8-a4f5-2492ac6bbdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:54.636885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11959a-224e-46f8-a4f5-2492ac6bbdaf None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:54.637163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11959a-224e-46f8-a4f5-2492ac6bbdaf None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:54.637435 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11959a-224e-46f8-a4f5-2492ac6bbdaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:54.649745 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e11959a-224e-46f8-a4f5-2492ac6bbdaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:54.651600 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e11959a-224e-46f8-a4f5-2492ac6bbdaf None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:54.666890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-9e11959a-224e-46f8-a4f5-2492ac6bbdaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:54.666890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:54.666890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:54.666890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:54.666890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:54.666890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:54.666890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:54.666890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:54.666890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:54.666890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:54.666890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:54.666890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:54.666890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:54.666890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:54.666890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:54.666890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:54.667891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:54.667891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:54.667891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:54.667891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:54.667891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:54.667891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:54.667891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:54.667891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:54.667891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:54.667891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:54.667891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:54.667891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:54.667891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:54.667891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:54.667891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:54.667891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:54.667891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:54.668840 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:54.668840 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:54.668840 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:54.672656 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 493/992] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:54.674132 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c6ffa-0e37-4f21-ab6a-a03b1ce0fe44 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:54.745205 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cc42a-51c1-4375-99c2-f188b5065ab4 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:54.745482 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cc42a-51c1-4375-99c2-f188b5065ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:54.745797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cc42a-51c1-4375-99c2-f188b5065ab4 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:54.747444 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 494/993] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:55:54.752493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81bbff-d83c-47d4-99cc-279407492a3d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:54.752771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81bbff-d83c-47d4-99cc-279407492a3d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:54.753039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81bbff-d83c-47d4-99cc-279407492a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:54.771562 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6d81bbff-d83c-47d4-99cc-279407492a3d None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:54.772775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6d81bbff-d83c-47d4-99cc-279407492a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:54.785215 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c6ffa-0e37-4f21-ab6a-a03b1ce0fe44 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:54.823275 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e101f3d-0fbb-4a92-9b04-9a00034ff7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:54.824010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e101f3d-0fbb-4a92-9b04-9a00034ff7ab None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:54.824361 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e101f3d-0fbb-4a92-9b04-9a00034ff7ab None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:54.824706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e101f3d-0fbb-4a92-9b04-9a00034ff7ab None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:54.825419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e101f3d-0fbb-4a92-9b04-9a00034ff7ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:54.825788 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e101f3d-0fbb-4a92-9b04-9a00034ff7ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:54.827195 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e101f3d-0fbb-4a92-9b04-9a00034ff7ab None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:54.834617 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:54.850986 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 500/994] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:54.850986 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e101f3d-0fbb-4a92-9b04-9a00034ff7ab None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:54.858521 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 495/995] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:54 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:55:54.873721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d81bbff-d83c-47d4-99cc-279407492a3d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:54.925048 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:54.953936 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e101f3d-0fbb-4a92-9b04-9a00034ff7ab None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:54.983090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d81bbff-d83c-47d4-99cc-279407492a3d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:54.992533 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550016e-dafb-4656-9bbd-6c2ea6fa19d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:54.993165 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550016e-dafb-4656-9bbd-6c2ea6fa19d1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:54.993436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550016e-dafb-4656-9bbd-6c2ea6fa19d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:54.993925 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550016e-dafb-4656-9bbd-6c2ea6fa19d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:55.002233 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6550016e-dafb-4656-9bbd-6c2ea6fa19d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:55.003727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6550016e-dafb-4656-9bbd-6c2ea6fa19d1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:55.015383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-6550016e-dafb-4656-9bbd-6c2ea6fa19d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:55.015383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:55.015383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:55.015383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:55.015383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:55.015383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:55.015383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:55.015383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:55.015383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:55.015383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:55.015383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:55.015383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:55.015383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:55.015383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:55.015383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:55.016313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:55.016313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:55.016313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:55.016313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:55.016313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:55.016313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:55.016313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:55.016313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:55.016313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:55.016313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:55.016313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:55.016313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:55.016313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:55.016313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:55.016313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:55.016313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:55.016313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:55.017217 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:55.017217 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:55.017217 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:55.017217 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:55.017217 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:55.017217 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:55.017217 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:55.017217 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:55.017217 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 501/996] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:55:55.022230 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550016e-dafb-4656-9bbd-6c2ea6fa19d1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:55.023346 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba87810-1363-4414-9764-d94db1235fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:55.024247 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba87810-1363-4414-9764-d94db1235fae None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:55.024524 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba87810-1363-4414-9764-d94db1235fae None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:55.024791 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba87810-1363-4414-9764-d94db1235fae None admin] PATH_INFO: `/v3/services/compute` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:55.025375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba87810-1363-4414-9764-d94db1235fae None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:55.027257 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba87810-1363-4414-9764-d94db1235fae None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:55.036744 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-2ba87810-1363-4414-9764-d94db1235fae None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 16:55:55.036744 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:55.036744 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:55:55.036744 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:55:55.036744 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:55:55.036744 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:55:55.036744 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:55:55.036744 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:55.036744 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 16:55:55.036744 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:55.036744 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:55:55.036744 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:55.036744 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:55.036744 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:55.036744 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:55.036744 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:55.036744 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:55.037804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:55.037804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:55.037804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:55.037804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:55.037804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:55.037804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:55.037804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:55:55.037804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:55:55.037804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:55:55.037804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:55:55.037804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:55.037804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:55.037804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:55.037804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:55.037804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:55.037804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:55.044295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:55.044295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:55.044295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:55.044295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:55.044295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:55.044295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:55.044295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:55.044295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:55.044295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:55.044295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:55.044295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:55:55.044295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:55.044295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 16:55:55.044295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:55.044295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 496/997] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:55:55.045713 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba87810-1363-4414-9764-d94db1235fae None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:55.087728 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:55.139172 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550016e-dafb-4656-9bbd-6c2ea6fa19d1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:55.153376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba87810-1363-4414-9764-d94db1235fae None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:55.167231 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:55.178747 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed567602-96c8-49f9-9ad5-2a701012b867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:55.179420 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed567602-96c8-49f9-9ad5-2a701012b867 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:55.179745 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed567602-96c8-49f9-9ad5-2a701012b867 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:55.182576 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed567602-96c8-49f9-9ad5-2a701012b867 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:55.182576 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed567602-96c8-49f9-9ad5-2a701012b867 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:55.182576 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed567602-96c8-49f9-9ad5-2a701012b867 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:55.183056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed567602-96c8-49f9-9ad5-2a701012b867 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:55.191167 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020655e-797f-4282-94eb-c4774c244014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:55.191776 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020655e-797f-4282-94eb-c4774c244014 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:55.192050 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020655e-797f-4282-94eb-c4774c244014 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:55.192314 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020655e-797f-4282-94eb-c4774c244014 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:55.193407 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d020655e-797f-4282-94eb-c4774c244014 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:55.193722 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d020655e-797f-4282-94eb-c4774c244014 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:55.195093 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d020655e-797f-4282-94eb-c4774c244014 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:55.201290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 502/998] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:55.202193 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed567602-96c8-49f9-9ad5-2a701012b867 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:55.206054 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 497/999] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 10 16:55:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:55.211592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020655e-797f-4282-94eb-c4774c244014 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:55.309489 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed567602-96c8-49f9-9ad5-2a701012b867 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:55.322332 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020655e-797f-4282-94eb-c4774c244014 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:55.328297 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:55.347114 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8000e-fc28-415a-96c6-8179b0f076a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:55.347721 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8000e-fc28-415a-96c6-8179b0f076a8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:55.347999 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8000e-fc28-415a-96c6-8179b0f076a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:55.348275 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8000e-fc28-415a-96c6-8179b0f076a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:55.349053 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e8000e-fc28-415a-96c6-8179b0f076a8 None admin] RBAC: Adding query filter params (role.id=c37c16d90ff44d80991a2e4bdb50432c, scope.project.id=d11f33af53374dffa61132642a7770f4, user.id=4f9e1767ec90435f80248a2930081265) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:55.349361 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e8000e-fc28-415a-96c6-8179b0f076a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:55.350854 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e8000e-fc28-415a-96c6-8179b0f076a8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:55.357632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 503/1000] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:55:55 2022] GET /identity/v3/role_assignments?user.id=4f9e1767ec90435f80248a2930081265&scope.project.id=d11f33af53374dffa61132642a7770f4&role.id=c37c16d90ff44d80991a2e4bdb50432c => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:55.369590 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43588052-821b-46f1-a5d4-e493c336e57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:55.370229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43588052-821b-46f1-a5d4-e493c336e57e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:55.370566 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43588052-821b-46f1-a5d4-e493c336e57e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:55.370838 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43588052-821b-46f1-a5d4-e493c336e57e None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:55.371492 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43588052-821b-46f1-a5d4-e493c336e57e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:55.371793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43588052-821b-46f1-a5d4-e493c336e57e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:55.373216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43588052-821b-46f1-a5d4-e493c336e57e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:55.382405 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 498/1001] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 10 16:55:55 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:55.384975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8000e-fc28-415a-96c6-8179b0f076a8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:55.497427 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8000e-fc28-415a-96c6-8179b0f076a8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:55.535686 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b5594-18db-4943-9f17-bc8c42f727b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:55.536295 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b5594-18db-4943-9f17-bc8c42f727b3 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:55.536567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b5594-18db-4943-9f17-bc8c42f727b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:55.536829 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b5594-18db-4943-9f17-bc8c42f727b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:55.537481 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b5594-18db-4943-9f17-bc8c42f727b3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:55.538873 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b5594-18db-4943-9f17-bc8c42f727b3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:55.655508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 504/1002] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 16:55:55 2022] POST /identity/v3/endpoints => generated 334 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:56.667804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130074b7-dd46-4990-b9d8-6a6ca237791d None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:56.668319 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130074b7-dd46-4990-b9d8-6a6ca237791d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:56.668319 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130074b7-dd46-4990-b9d8-6a6ca237791d None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:56.669856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 499/1003] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:55:56.675035 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3cf67-ca62-4de1-a887-6a70a2c4e911 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:56.675525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3cf67-ca62-4de1-a887-6a70a2c4e911 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:56.675605 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3cf67-ca62-4de1-a887-6a70a2c4e911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:56.694873 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-40f3cf67-ca62-4de1-a887-6a70a2c4e911 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:56.696099 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-40f3cf67-ca62-4de1-a887-6a70a2c4e911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:56.732594 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:56.779036 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 505/1004] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:56 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:55:56.801276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130074b7-dd46-4990-b9d8-6a6ca237791d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:56.846190 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:56.908794 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130074b7-dd46-4990-b9d8-6a6ca237791d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:56.945014 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b63d7-fd43-4aad-abe9-6bd98bd32af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:56.945647 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b63d7-fd43-4aad-abe9-6bd98bd32af5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:56.945920 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b63d7-fd43-4aad-abe9-6bd98bd32af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:56.946182 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b63d7-fd43-4aad-abe9-6bd98bd32af5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:56.946780 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513b63d7-fd43-4aad-abe9-6bd98bd32af5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:56.948741 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513b63d7-fd43-4aad-abe9-6bd98bd32af5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:56.958687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-513b63d7-fd43-4aad-abe9-6bd98bd32af5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:55:56.958687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:56.958687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:55:56.958687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:55:56.958687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:55:56.958687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:55:56.958687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:55:56.958687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:56.958687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:55:56.958687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:56.958687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:55:56.958687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:56.958687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:56.958687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:56.958687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:56.958687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:56.958687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:56.959721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:56.959721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:56.959721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:56.959721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:56.959721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:56.959721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:56.959721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:55:56.959721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:55:56.959721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:55:56.959721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:55:56.959721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:56.959721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:56.959721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:56.959721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:56.959721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:56.959721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:56.960587 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:56.960587 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:56.960587 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:56.960587 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:56.960587 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:56.960587 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:56.960587 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:56.960587 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:56.960587 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:56.960587 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:56.960587 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:55:56.960587 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:56.960587 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 16:55:56.960587 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:56.962436 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 500/1005] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 10 16:55:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:56.966146 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f3cf67-ca62-4de1-a887-6a70a2c4e911 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:57.071008 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f3cf67-ca62-4de1-a887-6a70a2c4e911 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:57.107749 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c41b19-95de-481e-acdc-54c1df1696cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:57.108354 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c41b19-95de-481e-acdc-54c1df1696cc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:57.108631 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c41b19-95de-481e-acdc-54c1df1696cc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:57.108901 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c41b19-95de-481e-acdc-54c1df1696cc None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:57.109531 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c41b19-95de-481e-acdc-54c1df1696cc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:57.109829 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c41b19-95de-481e-acdc-54c1df1696cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:57.111262 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c41b19-95de-481e-acdc-54c1df1696cc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:57.122046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 506/1006] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 10 16:55:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:57.123641 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b63d7-fd43-4aad-abe9-6bd98bd32af5 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:57.239364 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b63d7-fd43-4aad-abe9-6bd98bd32af5 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:57.241936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:57.279780 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb30d2-1595-4371-96f8-837cd30ab273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:57.280382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb30d2-1595-4371-96f8-837cd30ab273 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:57.280655 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb30d2-1595-4371-96f8-837cd30ab273 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:57.280961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb30d2-1595-4371-96f8-837cd30ab273 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:57.281791 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffb30d2-1595-4371-96f8-837cd30ab273 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:57.282093 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffb30d2-1595-4371-96f8-837cd30ab273 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:57.284053 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffb30d2-1595-4371-96f8-837cd30ab273 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:57.291774 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 501/1007] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 10 16:55:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:57.297316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c41b19-95de-481e-acdc-54c1df1696cc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:57.330166 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:57.419432 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c41b19-95de-481e-acdc-54c1df1696cc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:57.462334 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9588e-c4b4-49bb-88ca-eee1c551489b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:57.462978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9588e-c4b4-49bb-88ca-eee1c551489b None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:57.463767 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9588e-c4b4-49bb-88ca-eee1c551489b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:57.464035 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9588e-c4b4-49bb-88ca-eee1c551489b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:57.464800 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9588e-c4b4-49bb-88ca-eee1c551489b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:57.466176 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9588e-c4b4-49bb-88ca-eee1c551489b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:57.491848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8526b-280f-4ecd-a340-f857e4f7040e None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:57.492045 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8526b-280f-4ecd-a340-f857e4f7040e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:57.492307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8526b-280f-4ecd-a340-f857e4f7040e None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:57.493710 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 502/1008] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:55:57.499030 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ed628-f192-4977-ae67-19f435bd6127 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:57.499311 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ed628-f192-4977-ae67-19f435bd6127 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:57.499583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ed628-f192-4977-ae67-19f435bd6127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:57.518990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-322ed628-f192-4977-ae67-19f435bd6127 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:57.520257 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-322ed628-f192-4977-ae67-19f435bd6127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:57.553068 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 507/1009] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 16:55:57 2022] POST /identity/v3/endpoints => generated 325 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:55:57.634478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 503/1010] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:57 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:55:57.659613 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9588e-c4b4-49bb-88ca-eee1c551489b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:57.769902 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:57.785903 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9588e-c4b4-49bb-88ca-eee1c551489b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:57.834327 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c545de8-666e-4ac3-8b22-e5b1c002b884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:57.835045 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c545de8-666e-4ac3-8b22-e5b1c002b884 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:57.835732 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c545de8-666e-4ac3-8b22-e5b1c002b884 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:57.836386 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c545de8-666e-4ac3-8b22-e5b1c002b884 None admin] PATH_INFO: `/v3/services/image` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:57.837204 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c545de8-666e-4ac3-8b22-e5b1c002b884 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:57.842203 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c545de8-666e-4ac3-8b22-e5b1c002b884 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:57.852245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-8c545de8-666e-4ac3-8b22-e5b1c002b884 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 16:55:57.852245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:57.852245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:55:57.852245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:55:57.852245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:55:57.852245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:55:57.852245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:55:57.852245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:57.852245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 16:55:57.852245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:57.852245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:55:57.852245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:57.852245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:57.852245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:57.852245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:57.852245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:57.852245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:57.853290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:57.853290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:57.853290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:57.853290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:57.853290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:57.853290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:57.853290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:55:57.853290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:55:57.853290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:55:57.853290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:55:57.853290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:57.853290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:57.853290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:57.853290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:57.853290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:57.853290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:57.854199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:57.854199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:57.854199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:57.854199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:57.854199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:57.854199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:57.854199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:57.854199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:57.854199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:57.854199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:57.854199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:55:57.854199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:55:57.854199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 16:55:57.854199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:57.855700 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 508/1011] 10.222.0.45 () {56 vars in 1105 bytes} [Wed Aug 10 16:55:57 2022] GET /identity/v3/services/image => generated 86 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:57.860749 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ed628-f192-4977-ae67-19f435bd6127 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:57.940805 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0197ad-b225-495d-977f-60448c71ac5d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:57.941089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0197ad-b225-495d-977f-60448c71ac5d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:57.941361 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0197ad-b225-495d-977f-60448c71ac5d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:57.942884 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 509/1012] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:55:57.948686 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b256b0-b00b-449a-bdfb-a81af5d712af None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:57.948956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b256b0-b00b-449a-bdfb-a81af5d712af None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:57.949208 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b256b0-b00b-449a-bdfb-a81af5d712af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:57.971255 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-90b256b0-b00b-449a-bdfb-a81af5d712af None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:57.974252 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-90b256b0-b00b-449a-bdfb-a81af5d712af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:57.978787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:57.988609 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ed628-f192-4977-ae67-19f435bd6127 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:58.038242 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f1fab-8409-4ded-bb95-b37158288103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:58.038811 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f1fab-8409-4ded-bb95-b37158288103 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:58.039085 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f1fab-8409-4ded-bb95-b37158288103 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:58.039354 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f1fab-8409-4ded-bb95-b37158288103 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:58.040081 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f1fab-8409-4ded-bb95-b37158288103 None admin] RBAC: Adding query filter params (name=image) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:58.040284 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f1fab-8409-4ded-bb95-b37158288103 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:58.042181 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f1fab-8409-4ded-bb95-b37158288103 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:58.055751 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 504/1013] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:58.057664 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 510/1014] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:57 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:55:58.064415 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f1fab-8409-4ded-bb95-b37158288103 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:58.081267 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b256b0-b00b-449a-bdfb-a81af5d712af None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:58.174069 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f1fab-8409-4ded-bb95-b37158288103 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:58.177503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:58.199184 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b256b0-b00b-449a-bdfb-a81af5d712af None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:58.218331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db25e4-612c-47b6-a826-960b83d4d72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:58.218977 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db25e4-612c-47b6-a826-960b83d4d72a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:58.219300 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db25e4-612c-47b6-a826-960b83d4d72a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:58.219569 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db25e4-612c-47b6-a826-960b83d4d72a None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:58.220246 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db25e4-612c-47b6-a826-960b83d4d72a None admin] RBAC: Adding query filter params (type=image) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:58.220774 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db25e4-612c-47b6-a826-960b83d4d72a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:58.222181 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db25e4-612c-47b6-a826-960b83d4d72a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:58.227828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 505/1015] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:58 2022] GET /identity/v3/services?type=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:58.237615 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f7523d-9dec-4838-91bb-cd476c796d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:58.238681 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7523d-9dec-4838-91bb-cd476c796d9b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:58.239211 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7523d-9dec-4838-91bb-cd476c796d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:58.239689 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7523d-9dec-4838-91bb-cd476c796d9b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:58.253379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f7523d-9dec-4838-91bb-cd476c796d9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:55:58.258841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-51f7523d-9dec-4838-91bb-cd476c796d9b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 16:55:58.258841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:58.258841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:58.258841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:58.258841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:58.258841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:58.258841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:58.258841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:58.258841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:58.258841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:58.258841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:58.258841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:58.258841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:55:58.258841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:55:58.258841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:55:58.258841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:55:58.259988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:55:58.259988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:55:58.259988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:55:58.259988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:55:58.259988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:58.259988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:58.259988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:58.259988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:58.259988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:58.259988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:58.259988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:58.259988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:58.259988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:58.259988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:58.259988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:58.259988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:58.261498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:58.261498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:58.261498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:58.261498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:58.261498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:55:58.261498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:55:58.261498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:55:58.261498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:55:58.261498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:55:58.261498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:55:58.261498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 16:55:58.261498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:58.266123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 511/1016] 10.222.0.45 () {56 vars in 1120 bytes} [Wed Aug 10 16:55:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:55:58.268735 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db25e4-612c-47b6-a826-960b83d4d72a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:58.299046 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:58.384435 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db25e4-612c-47b6-a826-960b83d4d72a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:58.422127 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc80e16-52f6-4d4e-84e2-6a40f94e7ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:58.422773 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc80e16-52f6-4d4e-84e2-6a40f94e7ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:58.423049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc80e16-52f6-4d4e-84e2-6a40f94e7ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:58.423313 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc80e16-52f6-4d4e-84e2-6a40f94e7ee3 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:58.423972 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc80e16-52f6-4d4e-84e2-6a40f94e7ee3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:58.424266 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc80e16-52f6-4d4e-84e2-6a40f94e7ee3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:58.425605 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc80e16-52f6-4d4e-84e2-6a40f94e7ee3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:58.435940 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 506/1017] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Aug 10 16:55:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:58.441205 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f7523d-9dec-4838-91bb-cd476c796d9b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:58.558499 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f7523d-9dec-4838-91bb-cd476c796d9b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:58.595166 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30e264-6715-4171-8061-239c657f8e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:58.595873 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30e264-6715-4171-8061-239c657f8e26 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:58.596378 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30e264-6715-4171-8061-239c657f8e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:58.596654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30e264-6715-4171-8061-239c657f8e26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:58.610601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e30e264-6715-4171-8061-239c657f8e26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:58.614769 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e30e264-6715-4171-8061-239c657f8e26 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:58.615564 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:58.630709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2e30e264-6715-4171-8061-239c657f8e26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:58.630709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:58.630709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:58.630709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:58.630709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:58.630709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:58.630709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:58.630709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:58.630709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:58.630709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:58.630709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:58.630709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:58.630709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:58.630709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:58.630709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:58.630709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:58.631774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:58.631774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:58.631774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:58.631774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:58.631774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:58.631774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:58.631774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:58.631774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:58.631774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:58.631774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:58.631774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:58.631774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:58.631774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:58.631774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:58.631774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:58.631774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:58.631774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:58.632683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:58.632683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:58.632683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:58.633472 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 512/1018] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:58.638709 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc80e16-52f6-4d4e-84e2-6a40f94e7ee3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:58.732515 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:58.750480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc80e16-52f6-4d4e-84e2-6a40f94e7ee3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:58.790731 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e9ca3-c20d-4df7-81a4-770556bf6754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:58.792060 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e9ca3-c20d-4df7-81a4-770556bf6754 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:58.792351 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e9ca3-c20d-4df7-81a4-770556bf6754 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:58.792624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e9ca3-c20d-4df7-81a4-770556bf6754 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:58.793272 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e9ca3-c20d-4df7-81a4-770556bf6754 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:58.793616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e9ca3-c20d-4df7-81a4-770556bf6754 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:58.795039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e9ca3-c20d-4df7-81a4-770556bf6754 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:58.812529 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 507/1019] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:58.814294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30e264-6715-4171-8061-239c657f8e26 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:58.917956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30e264-6715-4171-8061-239c657f8e26 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:58.952325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca975fed-8319-4e19-8205-f3ad505626a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:58.952952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca975fed-8319-4e19-8205-f3ad505626a5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:58.953229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca975fed-8319-4e19-8205-f3ad505626a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:58.953500 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca975fed-8319-4e19-8205-f3ad505626a5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:58.961909 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca975fed-8319-4e19-8205-f3ad505626a5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:58.963581 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca975fed-8319-4e19-8205-f3ad505626a5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:58.975051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ca975fed-8319-4e19-8205-f3ad505626a5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:55:58.975051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:58.975051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:58.975051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:58.975051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:58.975051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:58.975051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:58.975051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:58.975051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:58.975051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:58.975051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:58.975051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:58.975051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:55:58.975051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:55:58.975051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:55:58.975051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:55:58.976017 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:58.976017 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:58.976017 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:55:58.976017 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:58.976017 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:55:58.976017 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:55:58.976017 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:58.976017 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:58.976017 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:58.976017 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:58.976017 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:58.976017 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:58.976017 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:58.976017 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:58.976017 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:58.976017 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:58.976017 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:58.976017 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:58.976956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:58.976956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:58.976956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:55:58.976956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:55:58.976956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:55:58.976956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:55:58.976956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:55:58.976956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:55:58.976956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:55:58.976956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:55:58.978682 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 513/1020] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 10 16:55:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:58.982097 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e9ca3-c20d-4df7-81a4-770556bf6754 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:59.086082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e9ca3-c20d-4df7-81a4-770556bf6754 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:59.121553 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90f77e-43ee-441a-ae01-9916ea06831a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:59.122212 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90f77e-43ee-441a-ae01-9916ea06831a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:59.122517 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90f77e-43ee-441a-ae01-9916ea06831a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:59.122787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90f77e-43ee-441a-ae01-9916ea06831a None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:59.123506 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90f77e-43ee-441a-ae01-9916ea06831a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:59.123805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90f77e-43ee-441a-ae01-9916ea06831a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:59.125213 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90f77e-43ee-441a-ae01-9916ea06831a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:59.144042 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 508/1021] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 10 16:55:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:59.149951 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca975fed-8319-4e19-8205-f3ad505626a5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:59.208848 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:59.254940 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca975fed-8319-4e19-8205-f3ad505626a5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:59.288665 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41402e-3736-4374-8dd3-8266195fc283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:59.289275 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41402e-3736-4374-8dd3-8266195fc283 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:59.289557 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41402e-3736-4374-8dd3-8266195fc283 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:59.289822 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41402e-3736-4374-8dd3-8266195fc283 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:59.359737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41402e-3736-4374-8dd3-8266195fc283 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:59.361215 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41402e-3736-4374-8dd3-8266195fc283 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:59.416675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:59.465526 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 514/1022] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:55:59 2022] GET /identity/v3/auth/tokens => generated 1873 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:55:59.467899 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90f77e-43ee-441a-ae01-9916ea06831a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:59.470806 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:59.567837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90f77e-43ee-441a-ae01-9916ea06831a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:59.602630 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ecbaa4-07b4-4878-b564-e94ff5023163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:59.603246 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecbaa4-07b4-4878-b564-e94ff5023163 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:59.603521 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecbaa4-07b4-4878-b564-e94ff5023163 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:59.607482 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecbaa4-07b4-4878-b564-e94ff5023163 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:59.619063 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ecbaa4-07b4-4878-b564-e94ff5023163 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:59.620859 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ecbaa4-07b4-4878-b564-e94ff5023163 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:59.635818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-b7ecbaa4-07b4-4878-b564-e94ff5023163 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:59.635818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:59.635818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:59.635818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:59.635818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:59.635818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:59.635818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:59.635818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:59.635818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:59.635818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:59.635818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:59.635818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:59.635818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:55:59.635818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:55:59.635818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:55:59.635818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:55:59.636816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:59.636816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:59.636816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:59.636816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:59.636816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:59.636816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:59.636816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:59.636816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:59.636816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:59.636816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:59.636816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:59.636816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:59.636816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:59.636816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:59.636816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:59.636816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:59.636816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:55:59.642912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:55:59.642912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:55:59.642912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:59.642912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 509/1023] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:55:59.643143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41402e-3736-4374-8dd3-8266195fc283 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:59.692366 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ab82d-1e26-48a4-b435-893952b3b83f None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:59.692694 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ab82d-1e26-48a4-b435-893952b3b83f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:59.692918 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ab82d-1e26-48a4-b435-893952b3b83f None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:59.694270 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 510/1024] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:55:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:55:59.700728 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc006f-41be-4cc6-ab3d-a467cb912525 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:59.701004 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc006f-41be-4cc6-ab3d-a467cb912525 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:59.701270 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc006f-41be-4cc6-ab3d-a467cb912525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:59.720379 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-13bc006f-41be-4cc6-ab3d-a467cb912525 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:55:59.721569 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-13bc006f-41be-4cc6-ab3d-a467cb912525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:55:59.753556 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41402e-3736-4374-8dd3-8266195fc283 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:59.777630 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:59.789769 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ce0c2-650e-4be3-8489-ec8c2ea7b32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:59.790499 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ce0c2-650e-4be3-8489-ec8c2ea7b32a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:59.790783 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ce0c2-650e-4be3-8489-ec8c2ea7b32a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:59.791049 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ce0c2-650e-4be3-8489-ec8c2ea7b32a None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:59.791901 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ce0c2-650e-4be3-8489-ec8c2ea7b32a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:55:59.792215 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ce0c2-650e-4be3-8489-ec8c2ea7b32a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:59.793590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ce0c2-650e-4be3-8489-ec8c2ea7b32a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:59.798391 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 511/1025] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:55:59 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:55:59.812772 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 515/1026] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:55:59.813769 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc006f-41be-4cc6-ab3d-a467cb912525 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:59.821329 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ce0c2-650e-4be3-8489-ec8c2ea7b32a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:59.839241 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:55:59.924990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc006f-41be-4cc6-ab3d-a467cb912525 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:59.925175 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ce0c2-650e-4be3-8489-ec8c2ea7b32a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:55:59.960016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb4c09-e80d-48c2-992a-9119930ea5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:59.960344 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214343ce-01c1-46d4-b8fa-70d55032ac98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:55:59.960639 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb4c09-e80d-48c2-992a-9119930ea5c0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:59.960901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214343ce-01c1-46d4-b8fa-70d55032ac98 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:55:59.961022 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb4c09-e80d-48c2-992a-9119930ea5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:59.961265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb4c09-e80d-48c2-992a-9119930ea5c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:59.961365 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214343ce-01c1-46d4-b8fa-70d55032ac98 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:55:59.961452 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214343ce-01c1-46d4-b8fa-70d55032ac98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:55:59.962268 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdb4c09-e80d-48c2-992a-9119930ea5c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:59.963774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdb4c09-e80d-48c2-992a-9119930ea5c0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:59.970226 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 516/1027] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:55:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:55:59.971158 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb4c09-e80d-48c2-992a-9119930ea5c0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:55:59.971503 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214343ce-01c1-46d4-b8fa-70d55032ac98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:55:59.972917 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214343ce-01c1-46d4-b8fa-70d55032ac98 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:55:59.986237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-214343ce-01c1-46d4-b8fa-70d55032ac98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:59.986237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:55:59.986237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:55:59.986237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:55:59.986237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:55:59.986237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:55:59.986237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:55:59.986237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:55:59.986237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:55:59.986237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:55:59.986237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:55:59.986237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:55:59.986237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:55:59.986237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:55:59.986237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:55:59.987771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:55:59.987771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:55:59.987771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:55:59.987771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:55:59.987771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:55:59.987771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:55:59.987771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:55:59.987771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:55:59.987771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:55:59.987771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:55:59.987771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:55:59.987771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:55:59.987771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:55:59.987771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:55:59.987771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:55:59.987771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:55:59.987771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:55:59.988830 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:55:59.988830 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:55:59.988830 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:55:59.988830 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:55:59.988830 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:55:59.988830 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:55:59.988830 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:55:59.988830 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:55:59.988830 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 512/1028] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:55:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:55:59.992956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214343ce-01c1-46d4-b8fa-70d55032ac98 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:00.042641 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:00.061437 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:00.080066 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb4c09-e80d-48c2-992a-9119930ea5c0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:00.110539 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214343ce-01c1-46d4-b8fa-70d55032ac98 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:00.118416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e74c26-af70-4ab3-999b-f6761266a59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:00.119024 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e74c26-af70-4ab3-999b-f6761266a59c None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:00.119353 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e74c26-af70-4ab3-999b-f6761266a59c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:00.120143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e74c26-af70-4ab3-999b-f6761266a59c None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:00.120835 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e74c26-af70-4ab3-999b-f6761266a59c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:00.122222 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e74c26-af70-4ab3-999b-f6761266a59c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:00.146523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6fdc5-3018-42e9-b3ac-63d3564a4035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:00.147165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6fdc5-3018-42e9-b3ac-63d3564a4035 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:00.147491 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6fdc5-3018-42e9-b3ac-63d3564a4035 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:00.147760 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6fdc5-3018-42e9-b3ac-63d3564a4035 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:00.148446 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6fdc5-3018-42e9-b3ac-63d3564a4035 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:00.149004 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6fdc5-3018-42e9-b3ac-63d3564a4035 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:00.150458 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6fdc5-3018-42e9-b3ac-63d3564a4035 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:00.168491 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 513/1029] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:55:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:56:00.169941 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6fdc5-3018-42e9-b3ac-63d3564a4035 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:00.205939 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 517/1030] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 10 16:55:59 2022] POST /identity/v3/projects => generated 311 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:56:00.277139 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80cad7-f123-4f51-bf50-db726bb1efea None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:00.277433 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80cad7-f123-4f51-bf50-db726bb1efea None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:00.277707 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80cad7-f123-4f51-bf50-db726bb1efea None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:00.279390 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 518/1031] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:56:00.284954 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684756a-0e78-441e-8dd6-7663a4d8a4ba None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:00.285181 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6fdc5-3018-42e9-b3ac-63d3564a4035 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:00.285393 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684756a-0e78-441e-8dd6-7663a4d8a4ba None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:00.285574 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684756a-0e78-441e-8dd6-7663a4d8a4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:00.288760 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:00.305700 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e684756a-0e78-441e-8dd6-7663a4d8a4ba None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:00.307755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e684756a-0e78-441e-8dd6-7663a4d8a4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:00.327027 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad25c7a-97b7-4555-98cf-b7c2596f9baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:00.327641 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad25c7a-97b7-4555-98cf-b7c2596f9baa None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:00.327913 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad25c7a-97b7-4555-98cf-b7c2596f9baa None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:00.328182 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad25c7a-97b7-4555-98cf-b7c2596f9baa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:00.328919 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad25c7a-97b7-4555-98cf-b7c2596f9baa None admin] RBAC: Adding query filter params (role.id=c37c16d90ff44d80991a2e4bdb50432c, scope.project.id=d11f33af53374dffa61132642a7770f4, user.id=615cc36e21cc493b823e6cee695ac308) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:00.329221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad25c7a-97b7-4555-98cf-b7c2596f9baa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:00.330746 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad25c7a-97b7-4555-98cf-b7c2596f9baa None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:00.337908 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 514/1032] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:56:00 2022] GET /identity/v3/role_assignments?user.id=615cc36e21cc493b823e6cee695ac308&scope.project.id=d11f33af53374dffa61132642a7770f4&role.id=c37c16d90ff44d80991a2e4bdb50432c => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:56:00.387044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 519/1033] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:56:00 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:56:00.410709 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad25c7a-97b7-4555-98cf-b7c2596f9baa None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:00.523457 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad25c7a-97b7-4555-98cf-b7c2596f9baa None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:00.558718 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1dc8d5-3371-45cb-84af-5ed7b0ca3d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:00.559320 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dc8d5-3371-45cb-84af-5ed7b0ca3d41 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:00.559594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dc8d5-3371-45cb-84af-5ed7b0ca3d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:00.559858 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dc8d5-3371-45cb-84af-5ed7b0ca3d41 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:00.560498 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1dc8d5-3371-45cb-84af-5ed7b0ca3d41 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:00.561934 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1dc8d5-3371-45cb-84af-5ed7b0ca3d41 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:00.673417 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 515/1034] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 10 16:56:00 2022] POST /identity/v3/services => generated 242 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:56:02.318973 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b967fe-c69e-4575-a230-4abcfd1125b6 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:02.319501 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b967fe-c69e-4575-a230-4abcfd1125b6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:02.319501 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b967fe-c69e-4575-a230-4abcfd1125b6 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:02.320903 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 520/1035] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:56:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:56:02.326376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9308ee1-d9ad-4744-80fa-aa2c9b359e22 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:02.326796 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9308ee1-d9ad-4744-80fa-aa2c9b359e22 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:02.326968 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9308ee1-d9ad-4744-80fa-aa2c9b359e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:02.345461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a9308ee1-d9ad-4744-80fa-aa2c9b359e22 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:02.346672 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a9308ee1-d9ad-4744-80fa-aa2c9b359e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:02.372152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:02.379009 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cdf67-6a7e-4219-a7d8-e800b305e6ca None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:02.379299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cdf67-6a7e-4219-a7d8-e800b305e6ca None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:02.379580 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cdf67-6a7e-4219-a7d8-e800b305e6ca None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:02.380987 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 521/1036] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:56:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:56:02.386327 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e8a1d-6a6c-4387-990b-704705af941d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:02.386640 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e8a1d-6a6c-4387-990b-704705af941d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:02.386911 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e8a1d-6a6c-4387-990b-704705af941d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:02.405269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0c9e8a1d-6a6c-4387-990b-704705af941d None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:02.406523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0c9e8a1d-6a6c-4387-990b-704705af941d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:02.426902 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 516/1037] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:56:02 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:56:02.451667 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9308ee1-d9ad-4744-80fa-aa2c9b359e22 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:02.468649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:02.477696 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 522/1038] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:56:02 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:56:02.502168 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e8a1d-6a6c-4387-990b-704705af941d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:02.558645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9308ee1-d9ad-4744-80fa-aa2c9b359e22 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:02.597518 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc1413-ad05-44c8-957d-d94def8d6473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:02.598150 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc1413-ad05-44c8-957d-d94def8d6473 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:02.598989 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc1413-ad05-44c8-957d-d94def8d6473 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:02.599264 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc1413-ad05-44c8-957d-d94def8d6473 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:02.600676 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc1413-ad05-44c8-957d-d94def8d6473 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:02.603120 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc1413-ad05-44c8-957d-d94def8d6473 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:02.612784 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 517/1039] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:56:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:56:02.613670 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc1413-ad05-44c8-957d-d94def8d6473 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:02.618488 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e8a1d-6a6c-4387-990b-704705af941d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:02.640144 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:02.656378 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098848ca-828a-49d8-864a-dc91c8ee0778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:02.665939 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098848ca-828a-49d8-864a-dc91c8ee0778 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:02.665939 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098848ca-828a-49d8-864a-dc91c8ee0778 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:02.665939 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098848ca-828a-49d8-864a-dc91c8ee0778 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:02.668784 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098848ca-828a-49d8-864a-dc91c8ee0778 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:56:02.673492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-098848ca-828a-49d8-864a-dc91c8ee0778 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 16:56:02.673492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:56:02.673492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:56:02.673492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:56:02.673492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:56:02.673492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:56:02.673492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:56:02.673492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:56:02.673492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:56:02.673492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:56:02.673492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:56:02.673492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:56:02.673492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:56:02.673492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:56:02.673492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:56:02.673492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:56:02.674459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:56:02.674459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:56:02.674459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:56:02.674459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:56:02.674459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:56:02.674459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:56:02.674459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:56:02.674459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:56:02.674459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:56:02.674459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:56:02.674459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:56:02.674459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:56:02.674459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:56:02.674459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:56:02.674459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:56:02.674459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:56:02.675307 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:56:02.675307 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:56:02.675307 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:56:02.675307 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:56:02.675307 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:56:02.675307 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:56:02.675307 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:56:02.675307 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:56:02.675307 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:56:02.675307 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:56:02.675307 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 16:56:02.675307 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:56:02.675307 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 523/1040] 10.222.0.45 () {56 vars in 1120 bytes} [Wed Aug 10 16:56:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:56:02.680477 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098848ca-828a-49d8-864a-dc91c8ee0778 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:02.722089 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc1413-ad05-44c8-957d-d94def8d6473 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:02.736679 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:02.755623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9e0c8-f487-426d-887b-0781b3d9aade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:02.756211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9e0c8-f487-426d-887b-0781b3d9aade None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:02.756482 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9e0c8-f487-426d-887b-0781b3d9aade None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:02.756748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9e0c8-f487-426d-887b-0781b3d9aade None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:02.757374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f9e0c8-f487-426d-887b-0781b3d9aade None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:02.758832 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f9e0c8-f487-426d-887b-0781b3d9aade None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:02.780990 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098848ca-828a-49d8-864a-dc91c8ee0778 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:02.813843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88ccd5-1e0d-4640-9be7-d3ee8fba233c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:02.814454 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88ccd5-1e0d-4640-9be7-d3ee8fba233c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:02.814726 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88ccd5-1e0d-4640-9be7-d3ee8fba233c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:02.814989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88ccd5-1e0d-4640-9be7-d3ee8fba233c None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:02.815635 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e88ccd5-1e0d-4640-9be7-d3ee8fba233c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:02.815928 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e88ccd5-1e0d-4640-9be7-d3ee8fba233c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:02.817276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e88ccd5-1e0d-4640-9be7-d3ee8fba233c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:02.825858 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 524/1041] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Aug 10 16:56:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:56:02.827054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15586b9f-02d2-432c-8d06-dff06a0f726f None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:02.827337 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15586b9f-02d2-432c-8d06-dff06a0f726f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:02.827610 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15586b9f-02d2-432c-8d06-dff06a0f726f None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:02.828824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 525/1042] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:56:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:56:02.831828 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15586b9f-02d2-432c-8d06-dff06a0f726f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:02.849521 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 518/1043] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 10 16:56:02 2022] POST /identity/v3/users => generated 285 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:56:02.850721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf4e9f-a9b7-4725-9372-a918fea6e4d1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:02.851004 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf4e9f-a9b7-4725-9372-a918fea6e4d1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:02.851273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf4e9f-a9b7-4725-9372-a918fea6e4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:02.872175 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f1cf4e9f-a9b7-4725-9372-a918fea6e4d1 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:02.873824 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f1cf4e9f-a9b7-4725-9372-a918fea6e4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:02.949180 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15586b9f-02d2-432c-8d06-dff06a0f726f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:02.954137 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 519/1044] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:56:02 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:56:02.969190 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:02.976520 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf4e9f-a9b7-4725-9372-a918fea6e4d1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:02.988063 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:02.991478 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec074e-df2f-4249-9325-c5c7e8bbd387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:02.992126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec074e-df2f-4249-9325-c5c7e8bbd387 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:02.992456 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec074e-df2f-4249-9325-c5c7e8bbd387 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:02.992733 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec074e-df2f-4249-9325-c5c7e8bbd387 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:03.006820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec074e-df2f-4249-9325-c5c7e8bbd387 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:03.008199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec074e-df2f-4249-9325-c5c7e8bbd387 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:03.025256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-deec074e-df2f-4249-9325-c5c7e8bbd387 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:56:03.025256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:56:03.025256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:56:03.025256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:56:03.025256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:56:03.025256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:56:03.025256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:56:03.025256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:56:03.025256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:56:03.025256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:56:03.025256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:56:03.025256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:56:03.025256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:56:03.025256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:56:03.025256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:56:03.025256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:56:03.026634 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:56:03.026634 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:56:03.026634 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:56:03.026634 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:56:03.026634 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:56:03.026634 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:56:03.026634 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:56:03.026634 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:56:03.026634 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:56:03.026634 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:56:03.026634 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:56:03.026634 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:56:03.026634 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:56:03.026634 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:56:03.026634 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:56:03.026634 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:56:03.026634 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:56:03.027553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:56:03.027553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:56:03.027553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:56:03.027553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 526/1045] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:56:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:56:03.032131 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec074e-df2f-4249-9325-c5c7e8bbd387 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:03.094594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf4e9f-a9b7-4725-9372-a918fea6e4d1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:03.132346 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e55f41-3325-43fd-80a5-9fb773f651a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:03.132985 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e55f41-3325-43fd-80a5-9fb773f651a3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:03.133264 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e55f41-3325-43fd-80a5-9fb773f651a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:03.133534 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e55f41-3325-43fd-80a5-9fb773f651a3 None admin] PATH_INFO: `/v3/services/image` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:03.134123 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e55f41-3325-43fd-80a5-9fb773f651a3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:03.135836 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e55f41-3325-43fd-80a5-9fb773f651a3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:03.145191 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec074e-df2f-4249-9325-c5c7e8bbd387 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:03.145349 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-67e55f41-3325-43fd-80a5-9fb773f651a3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 16:56:03.145349 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:56:03.145349 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:56:03.145349 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:56:03.145349 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:56:03.145349 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:56:03.145349 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:56:03.145349 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:56:03.145349 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 16:56:03.145349 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:56:03.145349 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:56:03.145349 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:56:03.145349 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:56:03.145349 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:56:03.145349 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:56:03.145349 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:56:03.145349 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:56:03.146316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:56:03.146316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:56:03.146316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:56:03.146316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:56:03.146316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:56:03.146316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:56:03.146316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:56:03.146316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:56:03.146316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:56:03.146316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:56:03.146316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:56:03.146316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:56:03.146316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:56:03.146316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:56:03.146316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:56:03.146316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:56:03.147211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:56:03.147211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:56:03.147211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:56:03.147211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:56:03.147211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:56:03.147211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:56:03.147211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:56:03.147211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:56:03.147211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:56:03.147211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:56:03.147211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:56:03.147211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:56:03.147211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 16:56:03.147211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:56:03.147211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 520/1046] 10.222.0.45 () {56 vars in 1105 bytes} [Wed Aug 10 16:56:02 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:56:03.153020 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e55f41-3325-43fd-80a5-9fb773f651a3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:03.184911 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70e4af-a000-48f7-bb37-32ee26b802ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:03.185524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70e4af-a000-48f7-bb37-32ee26b802ed None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:03.185892 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70e4af-a000-48f7-bb37-32ee26b802ed None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:03.186174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70e4af-a000-48f7-bb37-32ee26b802ed None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:03.186854 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70e4af-a000-48f7-bb37-32ee26b802ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:03.187188 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70e4af-a000-48f7-bb37-32ee26b802ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:03.188566 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70e4af-a000-48f7-bb37-32ee26b802ed None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:03.205417 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 527/1047] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:56:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:56:03.206474 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70e4af-a000-48f7-bb37-32ee26b802ed None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:03.237506 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:03.239243 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:03.271142 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e55f41-3325-43fd-80a5-9fb773f651a3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:03.307447 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a3b28-d511-40cc-8cc9-d0794cdcf5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:03.308052 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a3b28-d511-40cc-8cc9-d0794cdcf5ae None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:03.308328 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a3b28-d511-40cc-8cc9-d0794cdcf5ae None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:03.308597 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a3b28-d511-40cc-8cc9-d0794cdcf5ae None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:03.309219 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a3b28-d511-40cc-8cc9-d0794cdcf5ae None admin] RBAC: Adding query filter params (name=image) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:03.309516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a3b28-d511-40cc-8cc9-d0794cdcf5ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:03.310914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a3b28-d511-40cc-8cc9-d0794cdcf5ae None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:03.313200 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70e4af-a000-48f7-bb37-32ee26b802ed None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:03.315862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 521/1048] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:56:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:56:03.321371 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a3b28-d511-40cc-8cc9-d0794cdcf5ae None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:03.351945 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab1af40-136a-49de-a5a3-b68d49ff7fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:03.352573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1af40-136a-49de-a5a3-b68d49ff7fac None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:03.352852 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1af40-136a-49de-a5a3-b68d49ff7fac None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:03.353133 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1af40-136a-49de-a5a3-b68d49ff7fac None admin] PATH_INFO: `/v3/users/nova` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:03.360586 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1af40-136a-49de-a5a3-b68d49ff7fac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:03.362091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1af40-136a-49de-a5a3-b68d49ff7fac None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:03.374448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-bab1af40-136a-49de-a5a3-b68d49ff7fac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:56:03.374448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:56:03.374448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:56:03.374448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:56:03.374448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:56:03.374448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:56:03.374448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:56:03.374448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:56:03.374448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:56:03.374448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:56:03.374448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:56:03.374448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:56:03.374448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:56:03.374448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:56:03.374448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:56:03.374448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:56:03.375573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:56:03.375573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:56:03.375573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:56:03.375573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:56:03.375573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:56:03.375573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:56:03.375573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:56:03.375573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:56:03.375573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:56:03.375573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:56:03.375573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:56:03.375573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:56:03.375573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:56:03.375573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:56:03.375573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:56:03.375573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:56:03.375573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:56:03.375573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:56:03.376570 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:56:03.376570 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:56:03.376570 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:56:03.376570 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:56:03.376570 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:56:03.376570 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:56:03.376570 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:56:03.376570 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:56:03.376570 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:56:03.376570 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:56:03.376570 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 528/1049] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 10 16:56:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:56:03.381118 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab1af40-136a-49de-a5a3-b68d49ff7fac None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:03.431568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a3b28-d511-40cc-8cc9-d0794cdcf5ae None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:03.468329 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6ea56-7edf-4745-8185-6ebfadca881b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:03.468924 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6ea56-7edf-4745-8185-6ebfadca881b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:03.469237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6ea56-7edf-4745-8185-6ebfadca881b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:03.469468 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6ea56-7edf-4745-8185-6ebfadca881b None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:03.470084 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff6ea56-7edf-4745-8185-6ebfadca881b None admin] RBAC: Adding query filter params (type=image) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:03.470401 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff6ea56-7edf-4745-8185-6ebfadca881b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:03.471745 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff6ea56-7edf-4745-8185-6ebfadca881b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:03.473834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:03.477424 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 522/1050] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:56:03 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:56:03.478370 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:03.483618 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6ea56-7edf-4745-8185-6ebfadca881b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:03.491361 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab1af40-136a-49de-a5a3-b68d49ff7fac None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:03.528601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649dbc80-4999-4ddc-9057-7f48a303e084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:03.529494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dbc80-4999-4ddc-9057-7f48a303e084 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:03.529952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dbc80-4999-4ddc-9057-7f48a303e084 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:03.530455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dbc80-4999-4ddc-9057-7f48a303e084 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:03.531353 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649dbc80-4999-4ddc-9057-7f48a303e084 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:03.531935 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649dbc80-4999-4ddc-9057-7f48a303e084 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:03.533534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649dbc80-4999-4ddc-9057-7f48a303e084 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:03.585999 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6ea56-7edf-4745-8185-6ebfadca881b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:03.641201 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 529/1051] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 10 16:56:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:56:03.646863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649dbc80-4999-4ddc-9057-7f48a303e084 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:03.672754 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c089fc7-6ca8-43a2-bc98-f0b71cfaf01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:03.673361 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c089fc7-6ca8-43a2-bc98-f0b71cfaf01b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:03.673639 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c089fc7-6ca8-43a2-bc98-f0b71cfaf01b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:03.673911 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c089fc7-6ca8-43a2-bc98-f0b71cfaf01b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:03.674842 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c089fc7-6ca8-43a2-bc98-f0b71cfaf01b None admin] RBAC: Adding query filter params (interface=public, service_id=d17172470dd14392a03f9cb08510eb9d, region_id=RegionOne) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:03.674952 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c089fc7-6ca8-43a2-bc98-f0b71cfaf01b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:03.676271 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c089fc7-6ca8-43a2-bc98-f0b71cfaf01b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:03.681651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 523/1052] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Aug 10 16:56:03 2022] GET /identity/v3/endpoints?service_id=d17172470dd14392a03f9cb08510eb9d&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:56:03.687476 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c089fc7-6ca8-43a2-bc98-f0b71cfaf01b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:03.749455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:03.761655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649dbc80-4999-4ddc-9057-7f48a303e084 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:03.788249 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c089fc7-6ca8-43a2-bc98-f0b71cfaf01b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:03.797302 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792f658-d4b6-4da2-85b5-02e8555678b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:03.798299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792f658-d4b6-4da2-85b5-02e8555678b0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:03.798817 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792f658-d4b6-4da2-85b5-02e8555678b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:03.799333 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792f658-d4b6-4da2-85b5-02e8555678b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:03.811114 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4792f658-d4b6-4da2-85b5-02e8555678b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:03.812874 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4792f658-d4b6-4da2-85b5-02e8555678b0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:03.820963 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:03.821966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d859d-e789-4174-abaf-7c2e4c62ed8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:03.822706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d859d-e789-4174-abaf-7c2e4c62ed8a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:03.823066 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d859d-e789-4174-abaf-7c2e4c62ed8a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:03.823338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d859d-e789-4174-abaf-7c2e4c62ed8a None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:03.823935 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d859d-e789-4174-abaf-7c2e4c62ed8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:03.825276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d859d-e789-4174-abaf-7c2e4c62ed8a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:03.831152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4792f658-d4b6-4da2-85b5-02e8555678b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:56:03.831152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:56:03.831152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:56:03.831152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:56:03.831152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:56:03.831152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:56:03.831152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:56:03.831152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:56:03.831152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:56:03.831152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:56:03.831152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:56:03.831152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:56:03.831152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:56:03.831152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:56:03.831152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:56:03.831152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:56:03.851033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:56:03.851033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:56:03.851033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:56:03.851033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:56:03.851033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:56:03.851033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:56:03.851033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:56:03.851033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:56:03.851033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:56:03.851033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:56:03.851033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:56:03.851033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:56:03.851033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:56:03.851033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:56:03.851033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:56:03.851033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:56:03.851033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:56:03.852044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:56:03.852044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:56:03.852044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:56:03.852220 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 524/1053] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 10 16:56:03 2022] GET /identity/v3/services => generated 1957 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 16:56:03.852296 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 530/1054] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:56:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:56:03.852376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d859d-e789-4174-abaf-7c2e4c62ed8a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:03.954684 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d859d-e789-4174-abaf-7c2e4c62ed8a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:03.983474 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:03.991592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44d5e5-45c5-439b-866e-b9cf5d2f6e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:03.992238 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44d5e5-45c5-439b-866e-b9cf5d2f6e32 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:03.992514 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44d5e5-45c5-439b-866e-b9cf5d2f6e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:03.992780 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44d5e5-45c5-439b-866e-b9cf5d2f6e32 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:03.993421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be44d5e5-45c5-439b-866e-b9cf5d2f6e32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:03.993719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be44d5e5-45c5-439b-866e-b9cf5d2f6e32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:03.995654 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be44d5e5-45c5-439b-866e-b9cf5d2f6e32 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:04.008439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 525/1055] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:56:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:04.014132 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792f658-d4b6-4da2-85b5-02e8555678b0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:04.132655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792f658-d4b6-4da2-85b5-02e8555678b0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:04.169242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f62808-76a1-4f92-8961-c7206b88eda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:04.169873 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f62808-76a1-4f92-8961-c7206b88eda8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:04.170146 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f62808-76a1-4f92-8961-c7206b88eda8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:04.170446 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f62808-76a1-4f92-8961-c7206b88eda8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:04.177503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f62808-76a1-4f92-8961-c7206b88eda8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:04.178974 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f62808-76a1-4f92-8961-c7206b88eda8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:04.190163 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-a6f62808-76a1-4f92-8961-c7206b88eda8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:56:04.190163 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:56:04.190163 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:56:04.190163 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:56:04.190163 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:56:04.190163 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:56:04.190163 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:56:04.190163 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:56:04.190163 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:56:04.190163 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:56:04.190163 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:56:04.190163 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:56:04.190163 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:56:04.190163 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:56:04.190163 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:56:04.191312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:56:04.191312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:56:04.191312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:56:04.191312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:56:04.191312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:56:04.191312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:56:04.191312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:56:04.191312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:56:04.191312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:56:04.191312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:56:04.191312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:56:04.191312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:56:04.191312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:56:04.191312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:56:04.191312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:56:04.191312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:56:04.191312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:56:04.192224 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:56:04.192224 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:56:04.192224 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:56:04.192224 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:56:04.192224 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:56:04.192224 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:56:04.192224 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:56:04.192224 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:56:04.192224 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 531/1056] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:56:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:56:04.201145 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44d5e5-45c5-439b-866e-b9cf5d2f6e32 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:04.304849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44d5e5-45c5-439b-866e-b9cf5d2f6e32 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:04.347971 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e13d7-01af-4e9e-83f9-8d03f021136b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:04.348565 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e13d7-01af-4e9e-83f9-8d03f021136b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:04.348834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e13d7-01af-4e9e-83f9-8d03f021136b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:04.349096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e13d7-01af-4e9e-83f9-8d03f021136b None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:04.349856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e13d7-01af-4e9e-83f9-8d03f021136b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:04.350152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e13d7-01af-4e9e-83f9-8d03f021136b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:04.351605 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e13d7-01af-4e9e-83f9-8d03f021136b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:04.370603 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 526/1057] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:56:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:04.372326 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f62808-76a1-4f92-8961-c7206b88eda8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:04.436890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:04.481652 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f62808-76a1-4f92-8961-c7206b88eda8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:04.516408 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f971f2-96da-494d-b7e1-bb9b55b02f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:04.517059 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f971f2-96da-494d-b7e1-bb9b55b02f1e None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:04.517331 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f971f2-96da-494d-b7e1-bb9b55b02f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:04.517596 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f971f2-96da-494d-b7e1-bb9b55b02f1e None admin] PATH_INFO: `/v3/projects/d11f33af53374dffa61132642a7770f4/users/615cc36e21cc493b823e6cee695ac308/roles/c37c16d90ff44d80991a2e4bdb50432c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:04.519719 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f971f2-96da-494d-b7e1-bb9b55b02f1e None admin] RBAC: Authorizing `identity:create_grant(project_id=d11f33af53374dffa61132642a7770f4, user_id=615cc36e21cc493b823e6cee695ac308, role_id=c37c16d90ff44d80991a2e4bdb50432c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:04.521139 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f971f2-96da-494d-b7e1-bb9b55b02f1e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:04.593823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 532/1058] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 10 16:56:04 2022] PUT /identity/v3/projects/d11f33af53374dffa61132642a7770f4/users/615cc36e21cc493b823e6cee695ac308/roles/c37c16d90ff44d80991a2e4bdb50432c => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:04.967125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e890197-ed54-4d38-8f46-2ad3251b15a7 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:04.967648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e890197-ed54-4d38-8f46-2ad3251b15a7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:04.967648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e890197-ed54-4d38-8f46-2ad3251b15a7 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:04.968969 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 527/1059] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:56:04.975352 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec2e3b-010a-475b-b1c8-f6792ef19b2b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:04.975720 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec2e3b-010a-475b-b1c8-f6792ef19b2b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:04.975892 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec2e3b-010a-475b-b1c8-f6792ef19b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:04.994455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d9ec2e3b-010a-475b-b1c8-f6792ef19b2b None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:04.995639 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d9ec2e3b-010a-475b-b1c8-f6792ef19b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:05.064196 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 533/1060] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:56:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:56:05.087411 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e890197-ed54-4d38-8f46-2ad3251b15a7 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:05.141307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:05.195025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e890197-ed54-4d38-8f46-2ad3251b15a7 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:05.229456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12e1ac-8d89-4cd0-8456-c79cf49add4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:05.229997 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12e1ac-8d89-4cd0-8456-c79cf49add4f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:05.230273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12e1ac-8d89-4cd0-8456-c79cf49add4f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:05.230571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12e1ac-8d89-4cd0-8456-c79cf49add4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:05.241058 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b12e1ac-8d89-4cd0-8456-c79cf49add4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:56:05.245848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-2b12e1ac-8d89-4cd0-8456-c79cf49add4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:56:05.245848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:56:05.245848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:56:05.245848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:56:05.245848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:56:05.245848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:56:05.245848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:56:05.245848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:56:05.245848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:56:05.245848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:56:05.245848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:56:05.245848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:56:05.245848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:56:05.245848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:56:05.245848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:56:05.245848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:56:05.247001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:56:05.247001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:56:05.247001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:56:05.247001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:56:05.247001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:56:05.247001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:56:05.247001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:56:05.247001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:56:05.247001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:56:05.247001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:56:05.247001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:56:05.247001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:56:05.247001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:56:05.247001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:56:05.247001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:56:05.247001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:56:05.252990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:56:05.252990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:56:05.252990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:56:05.252990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:56:05.252990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:56:05.252990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:56:05.252990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:56:05.252990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:56:05.252990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:56:05.252990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:56:05.252990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:56:05.252990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:56:05.252990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 528/1061] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:56:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:56:05.255357 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec2e3b-010a-475b-b1c8-f6792ef19b2b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:05.306481 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:05.358572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec2e3b-010a-475b-b1c8-f6792ef19b2b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:05.393504 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200bd39-d11a-4a5a-8ec9-acfd41b4737b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:05.394177 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200bd39-d11a-4a5a-8ec9-acfd41b4737b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:05.394459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200bd39-d11a-4a5a-8ec9-acfd41b4737b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:05.394724 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200bd39-d11a-4a5a-8ec9-acfd41b4737b None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:05.395375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5200bd39-d11a-4a5a-8ec9-acfd41b4737b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:05.395669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5200bd39-d11a-4a5a-8ec9-acfd41b4737b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:05.397025 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5200bd39-d11a-4a5a-8ec9-acfd41b4737b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:05.405981 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 534/1062] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:56:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:05.411638 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12e1ac-8d89-4cd0-8456-c79cf49add4f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:05.517846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12e1ac-8d89-4cd0-8456-c79cf49add4f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:05.527675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:05.552656 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a20e4-881d-417b-836f-fece113c7096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:05.553331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a20e4-881d-417b-836f-fece113c7096 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:05.553603 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a20e4-881d-417b-836f-fece113c7096 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:05.553869 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a20e4-881d-417b-836f-fece113c7096 None admin] PATH_INFO: `/v3/users/e4f7eda94e994d91901a2301d315f11f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:05.569822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a20e4-881d-417b-836f-fece113c7096 None admin] RBAC: Authorizing `identity:get_user(user_id=e4f7eda94e994d91901a2301d315f11f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:05.571318 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a20e4-881d-417b-836f-fece113c7096 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:05.573271 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 529/1063] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:56:05 2022] GET /identity/v3/users/e4f7eda94e994d91901a2301d315f11f => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:05.579270 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200bd39-d11a-4a5a-8ec9-acfd41b4737b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:05.694499 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200bd39-d11a-4a5a-8ec9-acfd41b4737b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:05.705449 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:05.730083 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8c7a0-b9d0-4d28-95b3-4ddbdad8ec93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:05.730767 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8c7a0-b9d0-4d28-95b3-4ddbdad8ec93 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:05.731042 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8c7a0-b9d0-4d28-95b3-4ddbdad8ec93 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:05.731307 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8c7a0-b9d0-4d28-95b3-4ddbdad8ec93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:05.803034 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc8c7a0-b9d0-4d28-95b3-4ddbdad8ec93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:05.804392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc8c7a0-b9d0-4d28-95b3-4ddbdad8ec93 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:05.895461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b7eba-974e-4f2a-8451-52ba070dfd99 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:05.895874 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b7eba-974e-4f2a-8451-52ba070dfd99 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:05.896211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b7eba-974e-4f2a-8451-52ba070dfd99 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:05.897645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 530/1064] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:56:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:56:05.903610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad06ed3-2c0b-4b74-8c92-d46462db50fb None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:05.903893 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad06ed3-2c0b-4b74-8c92-d46462db50fb None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:05.904169 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad06ed3-2c0b-4b74-8c92-d46462db50fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:05.917154 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 535/1065] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:56:05 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:56:05.918388 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8c7a0-b9d0-4d28-95b3-4ddbdad8ec93 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:05.923505 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-dad06ed3-2c0b-4b74-8c92-d46462db50fb None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:05.924780 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dad06ed3-2c0b-4b74-8c92-d46462db50fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:05.929975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:05.997874 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 531/1066] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:56:05 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:56:06.020792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad06ed3-2c0b-4b74-8c92-d46462db50fb None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:06.027033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8c7a0-b9d0-4d28-95b3-4ddbdad8ec93 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:06.065833 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756eb065-a3e1-435a-b120-17312d3aea8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:06.066522 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756eb065-a3e1-435a-b120-17312d3aea8f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:06.066674 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756eb065-a3e1-435a-b120-17312d3aea8f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:06.066949 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756eb065-a3e1-435a-b120-17312d3aea8f None admin] PATH_INFO: `/v3/projects/81ddd161e82a494985ba1f88f2a92941` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:06.068147 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756eb065-a3e1-435a-b120-17312d3aea8f None admin] RBAC: Authorizing `identity:get_project(project_id=81ddd161e82a494985ba1f88f2a92941)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:06.069685 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756eb065-a3e1-435a-b120-17312d3aea8f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:06.071517 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 536/1067] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:56:05 2022] GET /identity/v3/projects/81ddd161e82a494985ba1f88f2a92941 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:06.077870 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756eb065-a3e1-435a-b120-17312d3aea8f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:06.130380 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:06.141631 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad06ed3-2c0b-4b74-8c92-d46462db50fb None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:06.173764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:06.180134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089cfcd4-b348-4c19-96d5-1c81ecebc91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:06.180838 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cfcd4-b348-4c19-96d5-1c81ecebc91a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:06.181115 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cfcd4-b348-4c19-96d5-1c81ecebc91a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:06.181388 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cfcd4-b348-4c19-96d5-1c81ecebc91a None admin] PATH_INFO: `/v3/services/image` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:06.182030 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089cfcd4-b348-4c19-96d5-1c81ecebc91a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:06.187919 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089cfcd4-b348-4c19-96d5-1c81ecebc91a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:06.192359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-089cfcd4-b348-4c19-96d5-1c81ecebc91a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 16:56:06.192359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:56:06.192359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:56:06.192359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:56:06.192359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:56:06.192359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:56:06.192359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:56:06.192359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:56:06.192359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 16:56:06.192359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:56:06.192359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:56:06.192359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:56:06.192359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:56:06.192359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:56:06.192359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:56:06.192359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:56:06.192359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:56:06.193352 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:56:06.193352 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:56:06.193352 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:56:06.193352 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:56:06.193352 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:56:06.193352 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:56:06.193352 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:56:06.193352 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:56:06.193352 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:56:06.193352 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:56:06.193352 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:56:06.193352 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:56:06.193352 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:56:06.193352 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:56:06.193352 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:56:06.193352 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:56:06.194236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:56:06.194236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:56:06.194236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:56:06.194236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:56:06.194236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:56:06.194236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:56:06.194236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:56:06.194236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:56:06.194236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:56:06.194236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:56:06.194236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:56:06.194236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:56:06.194236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 16:56:06.194236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:56:06.194236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 532/1068] 10.222.0.45 () {56 vars in 1105 bytes} [Wed Aug 10 16:56:06 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:56:06.199593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756eb065-a3e1-435a-b120-17312d3aea8f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:06.199915 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089cfcd4-b348-4c19-96d5-1c81ecebc91a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:06.237450 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8adc0-37cc-4eee-b1b0-cf02c3e34651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:06.238063 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8adc0-37cc-4eee-b1b0-cf02c3e34651 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:06.238338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8adc0-37cc-4eee-b1b0-cf02c3e34651 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:06.238642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8adc0-37cc-4eee-b1b0-cf02c3e34651 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:06.239364 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8adc0-37cc-4eee-b1b0-cf02c3e34651 None admin] RBAC: Adding query filter params (role.id=7554b353873942fab506de830188aba0, scope.project.id=81ddd161e82a494985ba1f88f2a92941, user.id=e4f7eda94e994d91901a2301d315f11f) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:06.239669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8adc0-37cc-4eee-b1b0-cf02c3e34651 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:06.241093 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8adc0-37cc-4eee-b1b0-cf02c3e34651 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:06.247245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 537/1069] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:56:06 2022] GET /identity/v3/role_assignments?user.id=e4f7eda94e994d91901a2301d315f11f&scope.project.id=81ddd161e82a494985ba1f88f2a92941&role.id=7554b353873942fab506de830188aba0 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:56:06.313982 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089cfcd4-b348-4c19-96d5-1c81ecebc91a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:06.351512 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53122f15-98eb-4100-9e48-cb1038839d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:06.352128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53122f15-98eb-4100-9e48-cb1038839d76 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:06.352410 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53122f15-98eb-4100-9e48-cb1038839d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:06.352680 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53122f15-98eb-4100-9e48-cb1038839d76 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:06.353313 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53122f15-98eb-4100-9e48-cb1038839d76 None admin] RBAC: Adding query filter params (name=image) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:06.353686 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53122f15-98eb-4100-9e48-cb1038839d76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:06.355357 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53122f15-98eb-4100-9e48-cb1038839d76 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:06.365713 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 533/1070] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:56:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:56:06.366836 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8adc0-37cc-4eee-b1b0-cf02c3e34651 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:06.472658 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8adc0-37cc-4eee-b1b0-cf02c3e34651 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:06.511084 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d42ad-4a63-4396-b811-3a0a53c13978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:06.511697 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d42ad-4a63-4396-b811-3a0a53c13978 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:06.511975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d42ad-4a63-4396-b811-3a0a53c13978 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:06.512242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d42ad-4a63-4396-b811-3a0a53c13978 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:06.512876 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d42ad-4a63-4396-b811-3a0a53c13978 None admin] RBAC: Adding query filter params (type=image) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:06.513174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d42ad-4a63-4396-b811-3a0a53c13978 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:06.514649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d42ad-4a63-4396-b811-3a0a53c13978 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:06.525100 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 538/1071] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:56:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:06.526079 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53122f15-98eb-4100-9e48-cb1038839d76 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:06.552646 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:06.641261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53122f15-98eb-4100-9e48-cb1038839d76 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:06.675495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30df53-0202-4aa5-a89f-05f47890ccff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:06.676098 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30df53-0202-4aa5-a89f-05f47890ccff None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:06.676374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30df53-0202-4aa5-a89f-05f47890ccff None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:06.676643 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30df53-0202-4aa5-a89f-05f47890ccff None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:06.677303 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30df53-0202-4aa5-a89f-05f47890ccff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:06.678721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30df53-0202-4aa5-a89f-05f47890ccff None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:06.719304 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc6711-48ec-4a90-ad52-16320fc8a55c None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:06.719591 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc6711-48ec-4a90-ad52-16320fc8a55c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:06.719861 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc6711-48ec-4a90-ad52-16320fc8a55c None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:06.721239 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 539/1072] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:56:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:56:06.726068 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b7415-8454-463f-b265-2d05a7a4bd64 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:06.726382 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b7415-8454-463f-b265-2d05a7a4bd64 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:06.726650 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b7415-8454-463f-b265-2d05a7a4bd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:06.740541 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:06.746632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c91b7415-8454-463f-b265-2d05a7a4bd64 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:06.747814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c91b7415-8454-463f-b265-2d05a7a4bd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:06.778519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 534/1073] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 16:56:06 2022] POST /identity/v3/endpoints => generated 327 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 16:56:06.830091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 540/1074] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:56:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:56:06.854393 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30df53-0202-4aa5-a89f-05f47890ccff None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:06.945032 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:06.968102 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30df53-0202-4aa5-a89f-05f47890ccff None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:07.004141 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da12d00-a513-4bb9-a973-aade331f3b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:07.004798 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da12d00-a513-4bb9-a973-aade331f3b2c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:07.005061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da12d00-a513-4bb9-a973-aade331f3b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:07.005294 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da12d00-a513-4bb9-a973-aade331f3b2c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:07.016999 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da12d00-a513-4bb9-a973-aade331f3b2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:56:07.021679 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-9da12d00-a513-4bb9-a973-aade331f3b2c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 16:56:07.021679 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:56:07.021679 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:56:07.021679 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:56:07.021679 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:56:07.021679 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:56:07.021679 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:56:07.021679 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:56:07.021679 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:56:07.021679 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:56:07.021679 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:56:07.021679 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:56:07.021679 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:56:07.021679 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:56:07.021679 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:56:07.021679 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:56:07.022895 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:56:07.022895 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:56:07.022895 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:56:07.022895 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:56:07.022895 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:56:07.022895 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:56:07.022895 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:56:07.022895 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:56:07.022895 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:56:07.022895 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:56:07.022895 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:56:07.022895 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:56:07.022895 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:56:07.022895 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:56:07.022895 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:56:07.022895 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:56:07.024040 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:56:07.024040 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:56:07.024040 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:56:07.024040 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:56:07.024040 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:56:07.024040 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:56:07.024040 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:56:07.024040 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:56:07.024040 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:56:07.024040 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:56:07.024040 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 16:56:07.024040 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:56:07.024040 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 535/1075] 10.222.0.45 () {56 vars in 1120 bytes} [Wed Aug 10 16:56:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:56:07.030379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b7415-8454-463f-b265-2d05a7a4bd64 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:07.133305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b7415-8454-463f-b265-2d05a7a4bd64 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:07.164638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:07.170345 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af7b62-8fc9-4eb9-887d-1bfcf529c8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:07.170990 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af7b62-8fc9-4eb9-887d-1bfcf529c8cd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:07.171268 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af7b62-8fc9-4eb9-887d-1bfcf529c8cd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:07.171534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af7b62-8fc9-4eb9-887d-1bfcf529c8cd None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:07.172197 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af7b62-8fc9-4eb9-887d-1bfcf529c8cd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:07.172496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af7b62-8fc9-4eb9-887d-1bfcf529c8cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:07.173860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af7b62-8fc9-4eb9-887d-1bfcf529c8cd None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:07.187193 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 541/1076] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Aug 10 16:56:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:07.188328 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da12d00-a513-4bb9-a973-aade331f3b2c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:07.291749 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da12d00-a513-4bb9-a973-aade331f3b2c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:07.326060 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f0fa4-c4e7-40fd-979d-b40bf585f2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:07.326719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f0fa4-c4e7-40fd-979d-b40bf585f2eb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:07.327000 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f0fa4-c4e7-40fd-979d-b40bf585f2eb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:07.327276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f0fa4-c4e7-40fd-979d-b40bf585f2eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:07.338693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:07.341104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f0fa4-c4e7-40fd-979d-b40bf585f2eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:07.342708 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f0fa4-c4e7-40fd-979d-b40bf585f2eb None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:07.357616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-744f0fa4-c4e7-40fd-979d-b40bf585f2eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:56:07.357616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:56:07.357616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:56:07.357616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:56:07.357616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:56:07.357616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:56:07.357616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:56:07.357616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:56:07.357616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:56:07.357616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:56:07.357616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:56:07.357616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:56:07.357616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:56:07.357616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:56:07.357616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:56:07.357616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:56:07.358855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:56:07.358855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:56:07.358855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:56:07.358855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:56:07.358855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:56:07.358855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:56:07.358855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:56:07.358855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:56:07.358855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:56:07.358855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:56:07.358855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:56:07.358855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:56:07.358855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:56:07.358855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:56:07.358855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:56:07.358855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:56:07.358855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:56:07.360007 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:56:07.360007 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:56:07.360007 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:56:07.361321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 536/1077] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:56:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:56:07.367182 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af7b62-8fc9-4eb9-887d-1bfcf529c8cd None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:07.473469 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af7b62-8fc9-4eb9-887d-1bfcf529c8cd None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:07.507823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe0047-7e27-45e8-805a-c4af4f15a761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:07.508426 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe0047-7e27-45e8-805a-c4af4f15a761 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:07.508700 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe0047-7e27-45e8-805a-c4af4f15a761 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:07.508972 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe0047-7e27-45e8-805a-c4af4f15a761 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:07.509607 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe0047-7e27-45e8-805a-c4af4f15a761 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:07.509939 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe0047-7e27-45e8-805a-c4af4f15a761 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:07.511349 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe0047-7e27-45e8-805a-c4af4f15a761 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:07.528030 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 542/1078] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:07.529262 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f0fa4-c4e7-40fd-979d-b40bf585f2eb None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:07.633792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f0fa4-c4e7-40fd-979d-b40bf585f2eb None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:07.667965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44caa326-48cc-4831-80be-cb7d9015e266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:07.668577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44caa326-48cc-4831-80be-cb7d9015e266 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:07.668855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44caa326-48cc-4831-80be-cb7d9015e266 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:07.669124 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44caa326-48cc-4831-80be-cb7d9015e266 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:07.676651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44caa326-48cc-4831-80be-cb7d9015e266 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:07.678097 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44caa326-48cc-4831-80be-cb7d9015e266 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:07.689849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-44caa326-48cc-4831-80be-cb7d9015e266 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:56:07.689849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:56:07.689849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:56:07.689849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:56:07.689849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:56:07.689849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:56:07.689849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:56:07.689849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:56:07.689849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:56:07.689849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:56:07.689849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:56:07.689849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:56:07.689849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:56:07.689849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:56:07.689849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:56:07.689849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:56:07.695834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:56:07.695834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:56:07.695834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:56:07.695834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:56:07.695834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:56:07.695834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:56:07.695834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:56:07.695834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:56:07.695834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:56:07.695834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:56:07.695834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:56:07.695834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:56:07.695834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:56:07.695834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:56:07.695834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:56:07.695834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:56:07.695834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:56:07.695834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:56:07.696821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:56:07.696821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:56:07.696821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:56:07.696821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:56:07.696821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:56:07.696821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:56:07.696821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:56:07.696821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:56:07.696821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 16:56:07.696821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:56:07.696821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 537/1079] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 10 16:56:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:56:07.697992 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe0047-7e27-45e8-805a-c4af4f15a761 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:07.760823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:07.806396 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe0047-7e27-45e8-805a-c4af4f15a761 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:07.848508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1ee1f-ab15-42eb-a73d-d97186ebecf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:07.849424 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1ee1f-ab15-42eb-a73d-d97186ebecf8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:07.849850 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1ee1f-ab15-42eb-a73d-d97186ebecf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:07.850265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1ee1f-ab15-42eb-a73d-d97186ebecf8 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:07.851188 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1ee1f-ab15-42eb-a73d-d97186ebecf8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:07.851655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1ee1f-ab15-42eb-a73d-d97186ebecf8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:07.853253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1ee1f-ab15-42eb-a73d-d97186ebecf8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:07.873569 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 543/1080] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 10 16:56:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:07.879234 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44caa326-48cc-4831-80be-cb7d9015e266 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:07.939216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:07.990538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44caa326-48cc-4831-80be-cb7d9015e266 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:08.027705 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408a892-28a5-4d2c-8ef2-2c6e356d26e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:08.028379 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408a892-28a5-4d2c-8ef2-2c6e356d26e5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:08.028615 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408a892-28a5-4d2c-8ef2-2c6e356d26e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:08.028820 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408a892-28a5-4d2c-8ef2-2c6e356d26e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:08.100621 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0408a892-28a5-4d2c-8ef2-2c6e356d26e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:08.102185 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0408a892-28a5-4d2c-8ef2-2c6e356d26e5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:08.157852 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:08.207935 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 538/1081] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:56:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 16:56:08.215099 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1ee1f-ab15-42eb-a73d-d97186ebecf8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:08.314654 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c19e1-7c23-4198-9aa6-6f8fc6d751fa None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:08.315075 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c19e1-7c23-4198-9aa6-6f8fc6d751fa None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:08.315465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c19e1-7c23-4198-9aa6-6f8fc6d751fa None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:08.317018 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 539/1082] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 16:56:08.321420 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1ee1f-ab15-42eb-a73d-d97186ebecf8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:08.321903 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e86c3f-0c69-439b-9994-28395670c796 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:08.322805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e86c3f-0c69-439b-9994-28395670c796 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:08.323753 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e86c3f-0c69-439b-9994-28395670c796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:08.336202 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:08.342461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-12e86c3f-0c69-439b-9994-28395670c796 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:08.343655 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-12e86c3f-0c69-439b-9994-28395670c796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:08.362458 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75925251-b3af-42f4-b904-235b2d3afaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:08.363094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75925251-b3af-42f4-b904-235b2d3afaf1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:08.363369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75925251-b3af-42f4-b904-235b2d3afaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:08.363633 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75925251-b3af-42f4-b904-235b2d3afaf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:08.375388 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75925251-b3af-42f4-b904-235b2d3afaf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:08.376827 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75925251-b3af-42f4-b904-235b2d3afaf1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:08.392115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-75925251-b3af-42f4-b904-235b2d3afaf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:56:08.392115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:56:08.392115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:56:08.392115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:56:08.392115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:56:08.392115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:56:08.392115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:56:08.392115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:56:08.392115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:56:08.392115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:56:08.392115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:56:08.392115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:56:08.392115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:56:08.392115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:56:08.392115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:56:08.392115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:56:08.397959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:56:08.397959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:56:08.397959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:56:08.397959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:56:08.397959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:56:08.397959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:56:08.397959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:56:08.397959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:56:08.397959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:56:08.397959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:56:08.397959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:56:08.397959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:56:08.397959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:56:08.397959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:56:08.397959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:56:08.397959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:56:08.397959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:56:08.399412 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:56:08.399412 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:56:08.399412 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:56:08.399412 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 544/1083] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:56:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:56:08.399412 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75925251-b3af-42f4-b904-235b2d3afaf1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:08.425114 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 540/1084] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:56:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:56:08.447813 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e86c3f-0c69-439b-9994-28395670c796 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:08.515334 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75925251-b3af-42f4-b904-235b2d3afaf1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:08.539932 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:08.553256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a22acd-fba7-416c-bb65-94f540d14229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:08.553894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a22acd-fba7-416c-bb65-94f540d14229 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:08.554151 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a22acd-fba7-416c-bb65-94f540d14229 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:08.554442 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a22acd-fba7-416c-bb65-94f540d14229 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:08.555078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a22acd-fba7-416c-bb65-94f540d14229 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:08.555378 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a22acd-fba7-416c-bb65-94f540d14229 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:08.556841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a22acd-fba7-416c-bb65-94f540d14229 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:08.559110 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e86c3f-0c69-439b-9994-28395670c796 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:08.575173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 545/1085] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:08.576246 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a22acd-fba7-416c-bb65-94f540d14229 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:08.601505 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb8c2ac-ccbd-4015-a2e9-9a86eed1387e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:08.602131 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8c2ac-ccbd-4015-a2e9-9a86eed1387e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:08.602435 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8c2ac-ccbd-4015-a2e9-9a86eed1387e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:08.603249 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8c2ac-ccbd-4015-a2e9-9a86eed1387e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:08.603359 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:08.621225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb8c2ac-ccbd-4015-a2e9-9a86eed1387e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:56:08.626095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-0cb8c2ac-ccbd-4015-a2e9-9a86eed1387e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:56:08.626095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:56:08.626095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:56:08.626095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:56:08.626095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:56:08.626095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:56:08.626095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:56:08.626095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:56:08.626095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:56:08.626095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:56:08.626095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:56:08.626095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:56:08.626095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:56:08.626095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:56:08.626095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:56:08.626095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:56:08.627090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:56:08.627090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:56:08.627090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:56:08.627090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:56:08.627090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:56:08.627090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:56:08.627090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:56:08.627090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:56:08.627090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:56:08.627090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:56:08.627090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:56:08.627090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:56:08.627090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:56:08.627090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:56:08.627090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:56:08.627090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:56:08.627985 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:56:08.627985 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:56:08.627985 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:56:08.627985 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:56:08.627985 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:56:08.627985 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:56:08.627985 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:56:08.627985 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:56:08.627985 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:56:08.627985 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:56:08.627985 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:56:08.627985 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:56:08.627985 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 541/1086] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:56:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:56:08.633411 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb8c2ac-ccbd-4015-a2e9-9a86eed1387e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:08.695561 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a22acd-fba7-416c-bb65-94f540d14229 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:08.733320 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03632c68-80d8-445d-817e-5e2cea88dbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:08.734009 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03632c68-80d8-445d-817e-5e2cea88dbac None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:08.734288 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03632c68-80d8-445d-817e-5e2cea88dbac None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:08.734583 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03632c68-80d8-445d-817e-5e2cea88dbac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:08.738542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb8c2ac-ccbd-4015-a2e9-9a86eed1387e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:08.742456 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03632c68-80d8-445d-817e-5e2cea88dbac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:08.743970 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03632c68-80d8-445d-817e-5e2cea88dbac None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:08.757446 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-03632c68-80d8-445d-817e-5e2cea88dbac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:56:08.757446 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:56:08.757446 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:56:08.757446 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:56:08.757446 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:56:08.757446 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:56:08.757446 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:56:08.757446 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:56:08.757446 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:56:08.757446 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:56:08.757446 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:56:08.757446 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:56:08.757446 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:56:08.757446 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:56:08.757446 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:56:08.758406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:56:08.758406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:56:08.758406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:56:08.758406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:56:08.758406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:56:08.758406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:56:08.758406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:56:08.758406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:56:08.758406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:56:08.758406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:56:08.758406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:56:08.758406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:56:08.758406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:56:08.758406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:56:08.758406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:56:08.758406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:56:08.758406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:56:08.764921 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:56:08.764921 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:56:08.764921 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:56:08.764921 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:56:08.764921 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:56:08.764921 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:56:08.764921 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:56:08.764921 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:56:08.764921 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 546/1087] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:56:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:56:08.765418 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03632c68-80d8-445d-817e-5e2cea88dbac None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:08.780935 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb98ae1-e8db-4e11-ac1d-3c0f140cb574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:08.781569 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb98ae1-e8db-4e11-ac1d-3c0f140cb574 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:08.781789 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb98ae1-e8db-4e11-ac1d-3c0f140cb574 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:08.782134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb98ae1-e8db-4e11-ac1d-3c0f140cb574 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:08.782802 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb98ae1-e8db-4e11-ac1d-3c0f140cb574 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:08.783098 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb98ae1-e8db-4e11-ac1d-3c0f140cb574 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:08.784452 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb98ae1-e8db-4e11-ac1d-3c0f140cb574 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:08.793141 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 542/1088] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:56:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:56:08.799087 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb98ae1-e8db-4e11-ac1d-3c0f140cb574 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:08.803292 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:08.864370 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:08.880100 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03632c68-80d8-445d-817e-5e2cea88dbac None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:08.913348 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb98ae1-e8db-4e11-ac1d-3c0f140cb574 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:08.914903 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d243b5-5e1f-4ca7-a2a0-50ec58e59f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:08.915510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d243b5-5e1f-4ca7-a2a0-50ec58e59f10 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:08.915797 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d243b5-5e1f-4ca7-a2a0-50ec58e59f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:08.916699 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d243b5-5e1f-4ca7-a2a0-50ec58e59f10 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:08.917589 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d243b5-5e1f-4ca7-a2a0-50ec58e59f10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:08.917893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d243b5-5e1f-4ca7-a2a0-50ec58e59f10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:08.919349 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d243b5-5e1f-4ca7-a2a0-50ec58e59f10 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:08.931242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 547/1089] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:56:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:08.931242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff606728-b63d-426d-9cc4-7df9eba517ac None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:08.931242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff606728-b63d-426d-9cc4-7df9eba517ac None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:08.931242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff606728-b63d-426d-9cc4-7df9eba517ac None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:08.931242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 548/1090] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:56:08.937463 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff606728-b63d-426d-9cc4-7df9eba517ac None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:08.951363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc9ba1-821f-4b26-a109-0a72e9f0cc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:08.951917 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc9ba1-821f-4b26-a109-0a72e9f0cc62 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:08.952273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc9ba1-821f-4b26-a109-0a72e9f0cc62 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:08.952461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc9ba1-821f-4b26-a109-0a72e9f0cc62 None admin] PATH_INFO: `/v3/users/e4f7eda94e994d91901a2301d315f11f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:08.953586 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc9ba1-821f-4b26-a109-0a72e9f0cc62 None admin] RBAC: Authorizing `identity:get_user(user_id=e4f7eda94e994d91901a2301d315f11f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:08.955058 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc9ba1-821f-4b26-a109-0a72e9f0cc62 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:08.956645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 543/1091] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:56:08 2022] GET /identity/v3/users/e4f7eda94e994d91901a2301d315f11f => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:56:08.957828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb087caa-159f-498d-a26c-8f6dba1aa0ec None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:08.958109 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb087caa-159f-498d-a26c-8f6dba1aa0ec None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:08.958398 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb087caa-159f-498d-a26c-8f6dba1aa0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:08.976566 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-eb087caa-159f-498d-a26c-8f6dba1aa0ec None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:08.977754 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-eb087caa-159f-498d-a26c-8f6dba1aa0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:09.039575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff606728-b63d-426d-9cc4-7df9eba517ac None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:09.044828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 544/1092] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:56:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:56:09.045785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb087caa-159f-498d-a26c-8f6dba1aa0ec None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:09.048279 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:09.083583 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f4824-5b04-4491-ade7-8a10a3a3e2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:09.084182 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f4824-5b04-4491-ade7-8a10a3a3e2ae None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:09.084455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f4824-5b04-4491-ade7-8a10a3a3e2ae None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:09.084723 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f4824-5b04-4491-ade7-8a10a3a3e2ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:09.085441 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f4824-5b04-4491-ade7-8a10a3a3e2ae None admin] RBAC: Adding query filter params (role.id=c37c16d90ff44d80991a2e4bdb50432c, scope.project.id=d11f33af53374dffa61132642a7770f4, user.id=615cc36e21cc493b823e6cee695ac308) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:09.085743 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f4824-5b04-4491-ade7-8a10a3a3e2ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:09.087231 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f4824-5b04-4491-ade7-8a10a3a3e2ae None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:09.093904 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 549/1093] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:56:08 2022] GET /identity/v3/role_assignments?user.id=615cc36e21cc493b823e6cee695ac308&scope.project.id=d11f33af53374dffa61132642a7770f4&role.id=c37c16d90ff44d80991a2e4bdb50432c => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:56:09.094666 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f4824-5b04-4491-ade7-8a10a3a3e2ae None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:09.118250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:09.155416 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb087caa-159f-498d-a26c-8f6dba1aa0ec None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:09.193760 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f488b4-97a4-43cb-88c5-2cd776dfd5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:09.194803 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f488b4-97a4-43cb-88c5-2cd776dfd5fb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:09.195085 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f488b4-97a4-43cb-88c5-2cd776dfd5fb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:09.195359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f488b4-97a4-43cb-88c5-2cd776dfd5fb None admin] PATH_INFO: `/v3/projects/81ddd161e82a494985ba1f88f2a92941` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:09.196777 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f488b4-97a4-43cb-88c5-2cd776dfd5fb None admin] RBAC: Authorizing `identity:get_project(project_id=81ddd161e82a494985ba1f88f2a92941)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:09.198773 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f488b4-97a4-43cb-88c5-2cd776dfd5fb None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:09.205661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 545/1094] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:56:09 2022] GET /identity/v3/projects/81ddd161e82a494985ba1f88f2a92941 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:09.206746 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f488b4-97a4-43cb-88c5-2cd776dfd5fb None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:09.232043 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f4824-5b04-4491-ade7-8a10a3a3e2ae None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:09.274395 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3081194a-f151-4d51-8065-682444165947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:09.274942 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081194a-f151-4d51-8065-682444165947 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:09.275220 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081194a-f151-4d51-8065-682444165947 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:09.275546 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081194a-f151-4d51-8065-682444165947 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:09.293095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:09.328575 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f488b4-97a4-43cb-88c5-2cd776dfd5fb None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:09.360621 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3081194a-f151-4d51-8065-682444165947 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:09.362095 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3081194a-f151-4d51-8065-682444165947 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:09.368701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c999cb-7c39-4dee-8a45-040b3006058f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:09.369927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c999cb-7c39-4dee-8a45-040b3006058f None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:09.370197 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c999cb-7c39-4dee-8a45-040b3006058f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:09.370598 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c999cb-7c39-4dee-8a45-040b3006058f None admin] PATH_INFO: `/v3/projects/81ddd161e82a494985ba1f88f2a92941/users/e4f7eda94e994d91901a2301d315f11f/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:09.373355 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c999cb-7c39-4dee-8a45-040b3006058f None admin] RBAC: Authorizing `identity:create_grant(project_id=81ddd161e82a494985ba1f88f2a92941, user_id=e4f7eda94e994d91901a2301d315f11f, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:09.374885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c999cb-7c39-4dee-8a45-040b3006058f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:09.375410 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:09.445568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 546/1095] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 10 16:56:09 2022] PUT /identity/v3/projects/81ddd161e82a494985ba1f88f2a92941/users/e4f7eda94e994d91901a2301d315f11f/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 16:56:09.488803 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 550/1096] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:56:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:56:09.496654 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c999cb-7c39-4dee-8a45-040b3006058f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:09.631268 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c999cb-7c39-4dee-8a45-040b3006058f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:09.660740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:09.666328 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02653a6-17d9-4c41-a8f8-1f75a0173721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:09.672532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02653a6-17d9-4c41-a8f8-1f75a0173721 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:09.672532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02653a6-17d9-4c41-a8f8-1f75a0173721 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:09.672532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02653a6-17d9-4c41-a8f8-1f75a0173721 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:09.677604 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02653a6-17d9-4c41-a8f8-1f75a0173721 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:09.679068 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02653a6-17d9-4c41-a8f8-1f75a0173721 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:09.695025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-f02653a6-17d9-4c41-a8f8-1f75a0173721 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:56:09.695025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:56:09.695025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:56:09.695025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:56:09.695025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:56:09.695025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:56:09.695025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:56:09.695025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:56:09.695025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:56:09.695025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:56:09.695025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:56:09.695025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:56:09.695025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:56:09.695025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:56:09.695025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:56:09.695025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:56:09.696047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:56:09.696047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:56:09.696047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:56:09.696047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:56:09.696047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:56:09.696047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:56:09.696047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:56:09.696047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:56:09.696047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:56:09.696047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:56:09.696047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:56:09.696047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:56:09.696047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:56:09.696047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:56:09.696047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:56:09.696047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:56:09.696047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:56:09.696959 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:56:09.696959 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:56:09.696959 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:56:09.697291 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 547/1097] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:56:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:56:09.702626 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3081194a-f151-4d51-8065-682444165947 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:09.801670 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3081194a-f151-4d51-8065-682444165947 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:09.811828 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:09.835808 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af0a9a-c7f0-40d5-8266-04e6e0efff29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:09.836407 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af0a9a-c7f0-40d5-8266-04e6e0efff29 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:09.836682 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af0a9a-c7f0-40d5-8266-04e6e0efff29 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:09.836951 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af0a9a-c7f0-40d5-8266-04e6e0efff29 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:09.837585 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60af0a9a-c7f0-40d5-8266-04e6e0efff29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:09.837884 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60af0a9a-c7f0-40d5-8266-04e6e0efff29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:09.839429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60af0a9a-c7f0-40d5-8266-04e6e0efff29 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:09.862385 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 551/1098] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:56:11.491575 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6da05-69f0-40ef-9159-a55a9263ad98 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:11.492112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6da05-69f0-40ef-9159-a55a9263ad98 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:11.492112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6da05-69f0-40ef-9159-a55a9263ad98 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:11.493489 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 548/1099] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:56:11.499288 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612bd9e-13fc-4d86-a364-54985398e13b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:11.499697 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612bd9e-13fc-4d86-a364-54985398e13b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:11.499815 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612bd9e-13fc-4d86-a364-54985398e13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:11.517867 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d612bd9e-13fc-4d86-a364-54985398e13b None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:11.519053 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d612bd9e-13fc-4d86-a364-54985398e13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:11.585137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 552/1100] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:56:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:56:11.609363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6da05-69f0-40ef-9159-a55a9263ad98 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:11.713306 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6da05-69f0-40ef-9159-a55a9263ad98 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:11.748975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91670875-6902-4370-b63b-56d1253a603c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:11.749507 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91670875-6902-4370-b63b-56d1253a603c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:11.749778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91670875-6902-4370-b63b-56d1253a603c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:11.750040 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91670875-6902-4370-b63b-56d1253a603c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:11.760595 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91670875-6902-4370-b63b-56d1253a603c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:56:11.768257 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-91670875-6902-4370-b63b-56d1253a603c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:56:11.768257 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:56:11.768257 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:56:11.768257 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:56:11.768257 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:56:11.768257 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:56:11.768257 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:56:11.768257 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:56:11.768257 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:56:11.768257 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:56:11.768257 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:56:11.768257 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:56:11.768257 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:56:11.768257 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:56:11.768257 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:56:11.768257 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:56:11.769245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:56:11.769245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:56:11.769245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:56:11.769245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:56:11.769245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:56:11.769245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:56:11.769245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:56:11.769245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:56:11.769245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:56:11.769245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:56:11.769245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:56:11.769245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:56:11.769245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:56:11.769245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:56:11.769245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:56:11.769245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:56:11.770286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:56:11.770286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:56:11.770286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:56:11.770286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:56:11.770286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:56:11.770286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:56:11.770286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:56:11.770286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:56:11.770286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:56:11.770286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:56:11.770286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:56:11.770286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:56:11.770286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 549/1101] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:56:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:56:11.776363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d612bd9e-13fc-4d86-a364-54985398e13b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:11.868887 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:11.872537 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d612bd9e-13fc-4d86-a364-54985398e13b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:11.905372 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33505017-77b3-445b-ac20-5a47646c3e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:11.906215 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33505017-77b3-445b-ac20-5a47646c3e0c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:11.906706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33505017-77b3-445b-ac20-5a47646c3e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:11.907342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33505017-77b3-445b-ac20-5a47646c3e0c None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:11.908024 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33505017-77b3-445b-ac20-5a47646c3e0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:11.908327 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33505017-77b3-445b-ac20-5a47646c3e0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:11.909694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33505017-77b3-445b-ac20-5a47646c3e0c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:11.918658 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 553/1102] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:56:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:11.924462 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91670875-6902-4370-b63b-56d1253a603c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:11.968985 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:12.026869 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91670875-6902-4370-b63b-56d1253a603c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:12.060511 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47cff88-6002-410e-9fc3-29c5c884507d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:12.061144 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47cff88-6002-410e-9fc3-29c5c884507d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:12.061420 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47cff88-6002-410e-9fc3-29c5c884507d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:12.061689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47cff88-6002-410e-9fc3-29c5c884507d None admin] PATH_INFO: `/v3/users/e4f7eda94e994d91901a2301d315f11f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:12.062848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47cff88-6002-410e-9fc3-29c5c884507d None admin] RBAC: Authorizing `identity:get_user(user_id=e4f7eda94e994d91901a2301d315f11f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:12.064495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47cff88-6002-410e-9fc3-29c5c884507d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:12.066157 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 550/1103] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:56:11 2022] GET /identity/v3/users/e4f7eda94e994d91901a2301d315f11f => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:12.071991 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33505017-77b3-445b-ac20-5a47646c3e0c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:12.178689 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33505017-77b3-445b-ac20-5a47646c3e0c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:12.220383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddba6e-df67-4e74-a421-f4617c76724f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:12.220987 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddba6e-df67-4e74-a421-f4617c76724f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:12.221266 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddba6e-df67-4e74-a421-f4617c76724f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:12.221536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddba6e-df67-4e74-a421-f4617c76724f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:12.248961 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:12.288631 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ddba6e-df67-4e74-a421-f4617c76724f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:12.290002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ddba6e-df67-4e74-a421-f4617c76724f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:12.387618 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 554/1104] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:56:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:56:12.393587 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47cff88-6002-410e-9fc3-29c5c884507d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:12.489597 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47cff88-6002-410e-9fc3-29c5c884507d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:12.500760 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:12.524090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bceed4-4a0f-4f2a-bd6b-9640fbe47487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:12.524815 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bceed4-4a0f-4f2a-bd6b-9640fbe47487 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:12.525074 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bceed4-4a0f-4f2a-bd6b-9640fbe47487 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:12.525345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bceed4-4a0f-4f2a-bd6b-9640fbe47487 None admin] PATH_INFO: `/v3/projects/81ddd161e82a494985ba1f88f2a92941` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:12.526763 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bceed4-4a0f-4f2a-bd6b-9640fbe47487 None admin] RBAC: Authorizing `identity:get_project(project_id=81ddd161e82a494985ba1f88f2a92941)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:12.528187 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bceed4-4a0f-4f2a-bd6b-9640fbe47487 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:12.529809 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 551/1105] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:56:12 2022] GET /identity/v3/projects/81ddd161e82a494985ba1f88f2a92941 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:12.535998 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddba6e-df67-4e74-a421-f4617c76724f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:12.593201 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:12.633978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddba6e-df67-4e74-a421-f4617c76724f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:12.667845 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85d8ba-b825-44df-ae4a-9fee05cb277e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:12.668443 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85d8ba-b825-44df-ae4a-9fee05cb277e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:12.668719 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85d8ba-b825-44df-ae4a-9fee05cb277e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:12.668988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85d8ba-b825-44df-ae4a-9fee05cb277e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:12.669703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85d8ba-b825-44df-ae4a-9fee05cb277e None admin] RBAC: Adding query filter params (role.id=7554b353873942fab506de830188aba0, scope.project.id=81ddd161e82a494985ba1f88f2a92941, user.id=e4f7eda94e994d91901a2301d315f11f) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:12.670007 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85d8ba-b825-44df-ae4a-9fee05cb277e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:12.671471 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85d8ba-b825-44df-ae4a-9fee05cb277e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:12.677083 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 555/1106] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:56:12 2022] GET /identity/v3/role_assignments?user.id=e4f7eda94e994d91901a2301d315f11f&scope.project.id=81ddd161e82a494985ba1f88f2a92941&role.id=7554b353873942fab506de830188aba0 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:14.719304 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed828c-4245-4aca-9bbb-aada9a8a3f27 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:14.719827 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed828c-4245-4aca-9bbb-aada9a8a3f27 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:14.719827 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed828c-4245-4aca-9bbb-aada9a8a3f27 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:14.721176 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 552/1107] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:56:14.726284 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d043a-260e-4b32-ae39-56cf530e1136 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:14.726720 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d043a-260e-4b32-ae39-56cf530e1136 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:14.726849 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d043a-260e-4b32-ae39-56cf530e1136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:14.745846 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f43d043a-260e-4b32-ae39-56cf530e1136 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:14.747071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f43d043a-260e-4b32-ae39-56cf530e1136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:14.813927 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 556/1108] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:56:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:56:14.839199 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed828c-4245-4aca-9bbb-aada9a8a3f27 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:14.940492 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed828c-4245-4aca-9bbb-aada9a8a3f27 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:14.974370 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb791eb-f02d-41d2-bb2a-94270b68c6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:14.974991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb791eb-f02d-41d2-bb2a-94270b68c6db None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:14.975265 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb791eb-f02d-41d2-bb2a-94270b68c6db None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:14.975533 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb791eb-f02d-41d2-bb2a-94270b68c6db None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:14.976629 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb791eb-f02d-41d2-bb2a-94270b68c6db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:14.978076 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb791eb-f02d-41d2-bb2a-94270b68c6db None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:14.980247 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 553/1109] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:56:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:14.986074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d043a-260e-4b32-ae39-56cf530e1136 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:15.039754 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:15.084240 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d043a-260e-4b32-ae39-56cf530e1136 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:15.118678 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f4baa-6451-4389-8710-22fada1e2658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:15.120027 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f4baa-6451-4389-8710-22fada1e2658 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:15.120296 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f4baa-6451-4389-8710-22fada1e2658 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:15.120562 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f4baa-6451-4389-8710-22fada1e2658 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:15.121197 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f4baa-6451-4389-8710-22fada1e2658 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:15.122644 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f4baa-6451-4389-8710-22fada1e2658 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:15.227289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 557/1110] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 10 16:56:14 2022] POST /identity/v3/users => generated 286 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:56:17.293021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8aa512-aab4-4f95-a1b3-e2da05b35268 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:17.293579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8aa512-aab4-4f95-a1b3-e2da05b35268 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:17.293579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8aa512-aab4-4f95-a1b3-e2da05b35268 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:17.295010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 554/1111] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:56:17.300361 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7764e-de89-4f62-a561-766154dd4c41 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:17.300802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7764e-de89-4f62-a561-766154dd4c41 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:17.300892 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7764e-de89-4f62-a561-766154dd4c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:17.319315 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-49a7764e-de89-4f62-a561-766154dd4c41 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:17.320516 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-49a7764e-de89-4f62-a561-766154dd4c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:17.387710 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 558/1112] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:56:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:56:17.411340 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8aa512-aab4-4f95-a1b3-e2da05b35268 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:17.458558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:17.515823 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8aa512-aab4-4f95-a1b3-e2da05b35268 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:17.550733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c6288-945b-408c-b79c-333eb2ccb83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:17.551272 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c6288-945b-408c-b79c-333eb2ccb83c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:17.551544 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c6288-945b-408c-b79c-333eb2ccb83c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:17.551808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c6288-945b-408c-b79c-333eb2ccb83c None admin] PATH_INFO: `/v3/roles/a3e5816caacc425c9a6dd381fe0eddf0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:17.553177 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c6288-945b-408c-b79c-333eb2ccb83c None admin] RBAC: Authorizing `identity:get_role(role_id=a3e5816caacc425c9a6dd381fe0eddf0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:17.554571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c6288-945b-408c-b79c-333eb2ccb83c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:17.555891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 555/1113] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:56:17 2022] GET /identity/v3/roles/a3e5816caacc425c9a6dd381fe0eddf0 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:17.561961 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7764e-de89-4f62-a561-766154dd4c41 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:17.602929 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:17.660902 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7764e-de89-4f62-a561-766154dd4c41 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:17.694080 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12062d-44cf-4bf1-a5d3-0768aeecabbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:17.694726 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12062d-44cf-4bf1-a5d3-0768aeecabbf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:17.694999 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12062d-44cf-4bf1-a5d3-0768aeecabbf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:17.695268 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12062d-44cf-4bf1-a5d3-0768aeecabbf None admin] PATH_INFO: `/v3/users/6304fb27bbc84927a0f00bb0db0dece3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:17.708947 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12062d-44cf-4bf1-a5d3-0768aeecabbf None admin] RBAC: Authorizing `identity:get_user(user_id=6304fb27bbc84927a0f00bb0db0dece3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:17.710419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12062d-44cf-4bf1-a5d3-0768aeecabbf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:17.712064 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 559/1114] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:56:17 2022] GET /identity/v3/users/6304fb27bbc84927a0f00bb0db0dece3 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:17.718536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c6288-945b-408c-b79c-333eb2ccb83c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:17.818825 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c6288-945b-408c-b79c-333eb2ccb83c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:17.836983 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:17.857784 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9e0a3-c278-48e2-9c9b-823d76b0d17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:17.858417 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9e0a3-c278-48e2-9c9b-823d76b0d17c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:17.858690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9e0a3-c278-48e2-9c9b-823d76b0d17c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:17.858964 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9e0a3-c278-48e2-9c9b-823d76b0d17c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:17.930972 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9e0a3-c278-48e2-9c9b-823d76b0d17c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:17.933453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9e0a3-c278-48e2-9c9b-823d76b0d17c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:18.034981 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 556/1115] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:56:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:56:18.041722 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12062d-44cf-4bf1-a5d3-0768aeecabbf None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:18.137973 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:18.141747 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12062d-44cf-4bf1-a5d3-0768aeecabbf None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:18.175266 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6804d-0aec-4a7c-9307-62e51f4431dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:18.175897 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6804d-0aec-4a7c-9307-62e51f4431dc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:18.176395 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6804d-0aec-4a7c-9307-62e51f4431dc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:18.176684 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6804d-0aec-4a7c-9307-62e51f4431dc None admin] PATH_INFO: `/v3/projects/81ddd161e82a494985ba1f88f2a92941` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:18.178184 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac6804d-0aec-4a7c-9307-62e51f4431dc None admin] RBAC: Authorizing `identity:get_project(project_id=81ddd161e82a494985ba1f88f2a92941)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:18.179730 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac6804d-0aec-4a7c-9307-62e51f4431dc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:18.181583 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 560/1116] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:56:18 2022] GET /identity/v3/projects/81ddd161e82a494985ba1f88f2a92941 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:18.188607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9e0a3-c278-48e2-9c9b-823d76b0d17c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:18.204151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:18.290390 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9e0a3-c278-48e2-9c9b-823d76b0d17c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:18.324457 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee28709-98a3-4fe7-b632-9834962f13a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:18.325127 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee28709-98a3-4fe7-b632-9834962f13a5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:18.325312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee28709-98a3-4fe7-b632-9834962f13a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:18.325578 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee28709-98a3-4fe7-b632-9834962f13a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:18.326290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee28709-98a3-4fe7-b632-9834962f13a5 None admin] RBAC: Adding query filter params (role.id=a3e5816caacc425c9a6dd381fe0eddf0, scope.project.id=81ddd161e82a494985ba1f88f2a92941, user.id=6304fb27bbc84927a0f00bb0db0dece3) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:18.326623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee28709-98a3-4fe7-b632-9834962f13a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:18.328049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee28709-98a3-4fe7-b632-9834962f13a5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:18.333345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 557/1117] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:56:18 2022] GET /identity/v3/role_assignments?user.id=6304fb27bbc84927a0f00bb0db0dece3&scope.project.id=81ddd161e82a494985ba1f88f2a92941&role.id=a3e5816caacc425c9a6dd381fe0eddf0 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:20.351565 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b3404-2739-474a-b7b0-e03efdae57d3 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:20.352132 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b3404-2739-474a-b7b0-e03efdae57d3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:20.352132 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b3404-2739-474a-b7b0-e03efdae57d3 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:20.353964 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 561/1118] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:56:20.359192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12010f1-f07a-4454-8f3f-58448a868160 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:20.359628 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12010f1-f07a-4454-8f3f-58448a868160 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:20.359743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12010f1-f07a-4454-8f3f-58448a868160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:20.377783 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b12010f1-f07a-4454-8f3f-58448a868160 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:20.378988 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b12010f1-f07a-4454-8f3f-58448a868160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:20.445229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 558/1119] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:56:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:56:20.467488 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b3404-2739-474a-b7b0-e03efdae57d3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:20.567359 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b3404-2739-474a-b7b0-e03efdae57d3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:20.600254 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9dfe76-19de-41a1-a304-24fe4e240466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:20.600870 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9dfe76-19de-41a1-a304-24fe4e240466 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:20.601146 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9dfe76-19de-41a1-a304-24fe4e240466 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:20.601414 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9dfe76-19de-41a1-a304-24fe4e240466 None admin] PATH_INFO: `/v3/roles/a3e5816caacc425c9a6dd381fe0eddf0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:20.602874 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9dfe76-19de-41a1-a304-24fe4e240466 None admin] RBAC: Authorizing `identity:get_role(role_id=a3e5816caacc425c9a6dd381fe0eddf0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:20.605198 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9dfe76-19de-41a1-a304-24fe4e240466 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:20.606559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 562/1120] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:56:20 2022] GET /identity/v3/roles/a3e5816caacc425c9a6dd381fe0eddf0 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:20.612399 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12010f1-f07a-4454-8f3f-58448a868160 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:20.622624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:20.708199 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12010f1-f07a-4454-8f3f-58448a868160 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:20.741745 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc646d9-33ef-4b1a-9467-5a326fe00ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:20.742382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc646d9-33ef-4b1a-9467-5a326fe00ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:20.742665 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc646d9-33ef-4b1a-9467-5a326fe00ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:20.742933 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc646d9-33ef-4b1a-9467-5a326fe00ce0 None admin] PATH_INFO: `/v3/users/6304fb27bbc84927a0f00bb0db0dece3` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:20.744751 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc646d9-33ef-4b1a-9467-5a326fe00ce0 None admin] RBAC: Authorizing `identity:get_user(user_id=6304fb27bbc84927a0f00bb0db0dece3)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:20.746205 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc646d9-33ef-4b1a-9467-5a326fe00ce0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:20.747822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 559/1121] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:56:20 2022] GET /identity/v3/users/6304fb27bbc84927a0f00bb0db0dece3 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:20.753611 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9dfe76-19de-41a1-a304-24fe4e240466 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:20.787559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:20.852266 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9dfe76-19de-41a1-a304-24fe4e240466 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:20.886880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30cb91-26cd-48b0-91fb-446a52a59eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:20.887430 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30cb91-26cd-48b0-91fb-446a52a59eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:20.887708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30cb91-26cd-48b0-91fb-446a52a59eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:20.887977 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30cb91-26cd-48b0-91fb-446a52a59eb5 None admin] PATH_INFO: `/v3/projects/81ddd161e82a494985ba1f88f2a92941` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:20.889089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30cb91-26cd-48b0-91fb-446a52a59eb5 None admin] RBAC: Authorizing `identity:get_project(project_id=81ddd161e82a494985ba1f88f2a92941)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:20.890544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30cb91-26cd-48b0-91fb-446a52a59eb5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:20.892178 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 563/1122] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:56:20 2022] GET /identity/v3/projects/81ddd161e82a494985ba1f88f2a92941 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:20.898617 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc646d9-33ef-4b1a-9467-5a326fe00ce0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:20.975395 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:20.998704 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc646d9-33ef-4b1a-9467-5a326fe00ce0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:21.031949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d4bd5-a6a2-49d5-af27-98d64731dd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:21.032597 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d4bd5-a6a2-49d5-af27-98d64731dd40 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:21.032871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d4bd5-a6a2-49d5-af27-98d64731dd40 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:21.033140 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d4bd5-a6a2-49d5-af27-98d64731dd40 None admin] PATH_INFO: `/v3/projects/81ddd161e82a494985ba1f88f2a92941/users/6304fb27bbc84927a0f00bb0db0dece3/roles/a3e5816caacc425c9a6dd381fe0eddf0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:21.035277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d4bd5-a6a2-49d5-af27-98d64731dd40 None admin] RBAC: Authorizing `identity:create_grant(project_id=81ddd161e82a494985ba1f88f2a92941, user_id=6304fb27bbc84927a0f00bb0db0dece3, role_id=a3e5816caacc425c9a6dd381fe0eddf0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:21.036691 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d4bd5-a6a2-49d5-af27-98d64731dd40 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:21.100616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 560/1123] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 10 16:56:20 2022] PUT /identity/v3/projects/81ddd161e82a494985ba1f88f2a92941/users/6304fb27bbc84927a0f00bb0db0dece3/roles/a3e5816caacc425c9a6dd381fe0eddf0 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:23.112745 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df088e43-cf5c-46c5-ad85-a183cb2b965b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:23.113293 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df088e43-cf5c-46c5-ad85-a183cb2b965b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:23.113293 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df088e43-cf5c-46c5-ad85-a183cb2b965b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:23.114650 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 564/1124] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:56:23.120403 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1c381-a64f-43d4-b581-5506ce46074c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:23.120808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1c381-a64f-43d4-b581-5506ce46074c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:23.120934 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1c381-a64f-43d4-b581-5506ce46074c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:23.139719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c9e1c381-a64f-43d4-b581-5506ce46074c None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:23.140910 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c9e1c381-a64f-43d4-b581-5506ce46074c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:23.206511 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 561/1125] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:56:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:56:23.230423 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df088e43-cf5c-46c5-ad85-a183cb2b965b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:23.332836 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df088e43-cf5c-46c5-ad85-a183cb2b965b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:23.334370 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:23.366479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ec350-8a03-478a-b418-68b854281d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:23.367024 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ec350-8a03-478a-b418-68b854281d77 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:23.367301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ec350-8a03-478a-b418-68b854281d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:23.367569 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ec350-8a03-478a-b418-68b854281d77 None admin] PATH_INFO: `/v3/roles/a3e5816caacc425c9a6dd381fe0eddf0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:23.369221 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ec350-8a03-478a-b418-68b854281d77 None admin] RBAC: Authorizing `identity:get_role(role_id=a3e5816caacc425c9a6dd381fe0eddf0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:23.370671 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ec350-8a03-478a-b418-68b854281d77 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:23.372048 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 565/1126] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:56:23 2022] GET /identity/v3/roles/a3e5816caacc425c9a6dd381fe0eddf0 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:23.377966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e1c381-a64f-43d4-b581-5506ce46074c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:23.442165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:23.476738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e1c381-a64f-43d4-b581-5506ce46074c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:23.509170 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3dc04-94fa-40e4-aac7-754e33ec9596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:23.510378 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3dc04-94fa-40e4-aac7-754e33ec9596 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:23.510892 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3dc04-94fa-40e4-aac7-754e33ec9596 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:23.511184 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3dc04-94fa-40e4-aac7-754e33ec9596 None admin] PATH_INFO: `/v3/users/6304fb27bbc84927a0f00bb0db0dece3` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:23.512858 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb3dc04-94fa-40e4-aac7-754e33ec9596 None admin] RBAC: Authorizing `identity:get_user(user_id=6304fb27bbc84927a0f00bb0db0dece3)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:23.514293 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb3dc04-94fa-40e4-aac7-754e33ec9596 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:23.515913 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 562/1127] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:56:23 2022] GET /identity/v3/users/6304fb27bbc84927a0f00bb0db0dece3 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:23.521804 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ec350-8a03-478a-b418-68b854281d77 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:23.621063 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ec350-8a03-478a-b418-68b854281d77 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:23.655243 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60244e82-1fd7-4dac-893b-caa5650f8d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:23.655837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60244e82-1fd7-4dac-893b-caa5650f8d9b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:23.656151 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60244e82-1fd7-4dac-893b-caa5650f8d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:23.656488 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60244e82-1fd7-4dac-893b-caa5650f8d9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:23.695047 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:23.723294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60244e82-1fd7-4dac-893b-caa5650f8d9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:23.724747 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60244e82-1fd7-4dac-893b-caa5650f8d9b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:23.823038 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 566/1128] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:56:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:56:23.829022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3dc04-94fa-40e4-aac7-754e33ec9596 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:23.926611 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3dc04-94fa-40e4-aac7-754e33ec9596 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:23.960659 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560864ff-7cfa-40e3-aaf3-5ecaaab897de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:23.961270 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560864ff-7cfa-40e3-aaf3-5ecaaab897de None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:23.961540 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560864ff-7cfa-40e3-aaf3-5ecaaab897de None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:23.961800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560864ff-7cfa-40e3-aaf3-5ecaaab897de None admin] PATH_INFO: `/v3/projects/81ddd161e82a494985ba1f88f2a92941` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:23.963458 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560864ff-7cfa-40e3-aaf3-5ecaaab897de None admin] RBAC: Authorizing `identity:get_project(project_id=81ddd161e82a494985ba1f88f2a92941)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:23.964876 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560864ff-7cfa-40e3-aaf3-5ecaaab897de None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:23.966505 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 563/1129] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:56:23 2022] GET /identity/v3/projects/81ddd161e82a494985ba1f88f2a92941 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:23.972702 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60244e82-1fd7-4dac-893b-caa5650f8d9b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:24.039947 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:24.070008 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60244e82-1fd7-4dac-893b-caa5650f8d9b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:24.104647 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8612705-2f26-4c68-a223-aee8b9cf4fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:24.105241 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8612705-2f26-4c68-a223-aee8b9cf4fca None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:24.105516 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8612705-2f26-4c68-a223-aee8b9cf4fca None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:24.105784 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8612705-2f26-4c68-a223-aee8b9cf4fca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:24.106527 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8612705-2f26-4c68-a223-aee8b9cf4fca None admin] RBAC: Adding query filter params (role.id=a3e5816caacc425c9a6dd381fe0eddf0, scope.project.id=81ddd161e82a494985ba1f88f2a92941, user.id=6304fb27bbc84927a0f00bb0db0dece3) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:24.106831 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8612705-2f26-4c68-a223-aee8b9cf4fca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:24.108248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8612705-2f26-4c68-a223-aee8b9cf4fca None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:24.113874 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 567/1130] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:56:23 2022] GET /identity/v3/role_assignments?user.id=6304fb27bbc84927a0f00bb0db0dece3&scope.project.id=81ddd161e82a494985ba1f88f2a92941&role.id=a3e5816caacc425c9a6dd381fe0eddf0 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:26.164547 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf226e2-a6af-4484-a46c-4e12ac91c1a4 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:26.165082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf226e2-a6af-4484-a46c-4e12ac91c1a4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:26.165082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf226e2-a6af-4484-a46c-4e12ac91c1a4 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:26.166553 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 564/1131] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:56:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:56:26.172269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de952888-5ae3-4fc4-bb73-9b9095571655 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:26.172705 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de952888-5ae3-4fc4-bb73-9b9095571655 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:26.172805 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de952888-5ae3-4fc4-bb73-9b9095571655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:26.191101 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-de952888-5ae3-4fc4-bb73-9b9095571655 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:26.192288 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-de952888-5ae3-4fc4-bb73-9b9095571655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:26.258664 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 568/1132] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:56:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:56:26.281980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf226e2-a6af-4484-a46c-4e12ac91c1a4 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:26.284932 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:26.382718 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf226e2-a6af-4484-a46c-4e12ac91c1a4 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:26.416163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ecb7f-aebe-49ff-b598-5cb963c7c696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:26.416708 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ecb7f-aebe-49ff-b598-5cb963c7c696 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:26.416979 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ecb7f-aebe-49ff-b598-5cb963c7c696 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:26.417244 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ecb7f-aebe-49ff-b598-5cb963c7c696 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:26.418382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ecb7f-aebe-49ff-b598-5cb963c7c696 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:26.419797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ecb7f-aebe-49ff-b598-5cb963c7c696 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:26.421367 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 565/1133] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:56:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:26.427404 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de952888-5ae3-4fc4-bb73-9b9095571655 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:26.487777 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:26.524882 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de952888-5ae3-4fc4-bb73-9b9095571655 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:26.558890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8545caa-6756-4704-9259-ca64801a6e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:26.559486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8545caa-6756-4704-9259-ca64801a6e4d None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:26.559765 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8545caa-6756-4704-9259-ca64801a6e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:26.560034 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8545caa-6756-4704-9259-ca64801a6e4d None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:26.560666 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8545caa-6756-4704-9259-ca64801a6e4d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:26.562101 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8545caa-6756-4704-9259-ca64801a6e4d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:26.644847 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 569/1134] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 10 16:56:26 2022] POST /identity/v3/projects => generated 311 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:56:28.690735 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eba67e-b144-4b97-b1c9-b4e1388c616e None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:28.691245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eba67e-b144-4b97-b1c9-b4e1388c616e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:28.691373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eba67e-b144-4b97-b1c9-b4e1388c616e None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:28.692623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 566/1135] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:56:28.698587 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c02419-72a6-4163-982c-dd9269b5cef2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:28.699068 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c02419-72a6-4163-982c-dd9269b5cef2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:28.699180 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c02419-72a6-4163-982c-dd9269b5cef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:28.717562 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d4c02419-72a6-4163-982c-dd9269b5cef2 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:28.718796 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d4c02419-72a6-4163-982c-dd9269b5cef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:28.784305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 570/1136] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:56:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:56:28.808802 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eba67e-b144-4b97-b1c9-b4e1388c616e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:28.877918 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:28.908891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eba67e-b144-4b97-b1c9-b4e1388c616e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:28.942119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6443cb-f9c5-48ec-a174-56e6579aeb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:28.942755 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6443cb-f9c5-48ec-a174-56e6579aeb49 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:28.943031 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6443cb-f9c5-48ec-a174-56e6579aeb49 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:28.943296 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6443cb-f9c5-48ec-a174-56e6579aeb49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:28.944396 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6443cb-f9c5-48ec-a174-56e6579aeb49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:28.945812 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6443cb-f9c5-48ec-a174-56e6579aeb49 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:28.947986 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 567/1137] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:56:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:28.953808 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c02419-72a6-4163-982c-dd9269b5cef2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:28.999153 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:29.049855 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c02419-72a6-4163-982c-dd9269b5cef2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:29.083742 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40577cdf-49c6-41de-84ae-3e773fe06752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:29.084337 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40577cdf-49c6-41de-84ae-3e773fe06752 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:29.084613 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40577cdf-49c6-41de-84ae-3e773fe06752 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:29.084882 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40577cdf-49c6-41de-84ae-3e773fe06752 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:29.085512 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40577cdf-49c6-41de-84ae-3e773fe06752 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:29.086980 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40577cdf-49c6-41de-84ae-3e773fe06752 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:29.209073 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 571/1138] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 10 16:56:28 2022] POST /identity/v3/users => generated 286 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:56:31.292166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598a08c-c143-4db7-98b9-7eb19ddb14fb None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:31.292696 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598a08c-c143-4db7-98b9-7eb19ddb14fb None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:31.292696 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598a08c-c143-4db7-98b9-7eb19ddb14fb None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:31.294048 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 568/1139] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:56:31.299279 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a42d25-022b-48e1-8f28-11d739b8a8d3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:31.299698 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a42d25-022b-48e1-8f28-11d739b8a8d3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:31.299823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a42d25-022b-48e1-8f28-11d739b8a8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:31.318157 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d3a42d25-022b-48e1-8f28-11d739b8a8d3 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:31.319372 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d3a42d25-022b-48e1-8f28-11d739b8a8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:31.385881 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 572/1140] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:56:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:56:31.409650 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598a08c-c143-4db7-98b9-7eb19ddb14fb None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:31.511956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598a08c-c143-4db7-98b9-7eb19ddb14fb None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:31.545951 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f491bc-30c7-4f8f-85ef-4ff06b72cdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:31.546553 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f491bc-30c7-4f8f-85ef-4ff06b72cdce None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:31.546830 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f491bc-30c7-4f8f-85ef-4ff06b72cdce None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:31.547097 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f491bc-30c7-4f8f-85ef-4ff06b72cdce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:31.554018 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:31.558639 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f491bc-30c7-4f8f-85ef-4ff06b72cdce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:56:31.563363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-41f491bc-30c7-4f8f-85ef-4ff06b72cdce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:56:31.563363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:56:31.563363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:56:31.563363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:56:31.563363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:56:31.563363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:56:31.563363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:56:31.563363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:56:31.563363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:56:31.563363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:56:31.563363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:56:31.563363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:56:31.563363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:56:31.563363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:56:31.563363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:56:31.563363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:56:31.574453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:56:31.574453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:56:31.574453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:56:31.574453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:56:31.574453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:56:31.574453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:56:31.574453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:56:31.574453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:56:31.574453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:56:31.574453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:56:31.574453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:56:31.574453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:56:31.574453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:56:31.574453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:56:31.574453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:56:31.574453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:56:31.575570 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:56:31.575570 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:56:31.575570 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:56:31.575570 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:56:31.575570 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:56:31.575570 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:56:31.575570 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:56:31.575570 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:56:31.575570 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:56:31.575570 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:56:31.575570 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:56:31.575570 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:56:31.575570 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 569/1141] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:56:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:56:31.576473 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a42d25-022b-48e1-8f28-11d739b8a8d3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:31.601474 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:31.672585 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a42d25-022b-48e1-8f28-11d739b8a8d3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:31.706075 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294736f-bec9-4258-879f-feb95f148836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:31.706700 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294736f-bec9-4258-879f-feb95f148836 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:31.706977 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294736f-bec9-4258-879f-feb95f148836 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:31.707245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294736f-bec9-4258-879f-feb95f148836 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:31.707904 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0294736f-bec9-4258-879f-feb95f148836 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:31.708202 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0294736f-bec9-4258-879f-feb95f148836 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:31.709566 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0294736f-bec9-4258-879f-feb95f148836 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:31.718168 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 573/1142] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:56:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:31.724007 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f491bc-30c7-4f8f-85ef-4ff06b72cdce None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:31.823154 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f491bc-30c7-4f8f-85ef-4ff06b72cdce None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:31.856707 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8344d4a-b60b-4906-ac9e-1516e68b3f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:31.857308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8344d4a-b60b-4906-ac9e-1516e68b3f75 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:31.857583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8344d4a-b60b-4906-ac9e-1516e68b3f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:31.857848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8344d4a-b60b-4906-ac9e-1516e68b3f75 None admin] PATH_INFO: `/v3/users/01d00ebfebb84c5d96b98060c3784788` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:31.870806 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8344d4a-b60b-4906-ac9e-1516e68b3f75 None admin] RBAC: Authorizing `identity:get_user(user_id=01d00ebfebb84c5d96b98060c3784788)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:31.872248 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8344d4a-b60b-4906-ac9e-1516e68b3f75 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:31.873900 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 570/1143] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:56:31 2022] GET /identity/v3/users/01d00ebfebb84c5d96b98060c3784788 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:31.879964 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294736f-bec9-4258-879f-feb95f148836 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:31.972138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:31.979413 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294736f-bec9-4258-879f-feb95f148836 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:32.012952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe70cd-93f5-45d3-ab4f-12577f8c5420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:32.013556 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe70cd-93f5-45d3-ab4f-12577f8c5420 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:32.013835 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe70cd-93f5-45d3-ab4f-12577f8c5420 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:32.014106 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe70cd-93f5-45d3-ab4f-12577f8c5420 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:32.080748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe70cd-93f5-45d3-ab4f-12577f8c5420 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:32.082117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe70cd-93f5-45d3-ab4f-12577f8c5420 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:32.175538 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:32.183353 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 574/1144] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:56:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:56:32.189311 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8344d4a-b60b-4906-ac9e-1516e68b3f75 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:32.249307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:32.293410 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8344d4a-b60b-4906-ac9e-1516e68b3f75 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:32.327806 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dfa3d6-0c49-4264-ae91-395616830637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:32.328447 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfa3d6-0c49-4264-ae91-395616830637 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:32.328710 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfa3d6-0c49-4264-ae91-395616830637 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:32.328978 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfa3d6-0c49-4264-ae91-395616830637 None admin] PATH_INFO: `/v3/projects/ff63ef613ad34164a1bb27a0b3f0ef29` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:32.330206 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dfa3d6-0c49-4264-ae91-395616830637 None admin] RBAC: Authorizing `identity:get_project(project_id=ff63ef613ad34164a1bb27a0b3f0ef29)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:32.331697 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dfa3d6-0c49-4264-ae91-395616830637 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:32.333348 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 571/1145] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:56:32 2022] GET /identity/v3/projects/ff63ef613ad34164a1bb27a0b3f0ef29 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:32.340314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe70cd-93f5-45d3-ab4f-12577f8c5420 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:32.446129 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe70cd-93f5-45d3-ab4f-12577f8c5420 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:32.480588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d026a5-6ee4-4c50-996e-674aae04c3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:32.481183 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d026a5-6ee4-4c50-996e-674aae04c3fb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:32.481457 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d026a5-6ee4-4c50-996e-674aae04c3fb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:32.481722 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d026a5-6ee4-4c50-996e-674aae04c3fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:32.482466 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d026a5-6ee4-4c50-996e-674aae04c3fb None admin] RBAC: Adding query filter params (role.id=7554b353873942fab506de830188aba0, scope.project.id=ff63ef613ad34164a1bb27a0b3f0ef29, user.id=01d00ebfebb84c5d96b98060c3784788) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:32.482769 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d026a5-6ee4-4c50-996e-674aae04c3fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:32.484191 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d026a5-6ee4-4c50-996e-674aae04c3fb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:32.489492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 575/1146] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:56:32 2022] GET /identity/v3/role_assignments?user.id=01d00ebfebb84c5d96b98060c3784788&scope.project.id=ff63ef613ad34164a1bb27a0b3f0ef29&role.id=7554b353873942fab506de830188aba0 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:34.530555 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d993e9-6d90-49b3-84e0-ddaee3fb907b None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:34.531132 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d993e9-6d90-49b3-84e0-ddaee3fb907b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:34.531132 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d993e9-6d90-49b3-84e0-ddaee3fb907b None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:34.532441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 572/1147] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:56:34.537436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd38739-e1c9-43de-afa4-457213ff1e3a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:34.537839 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd38739-e1c9-43de-afa4-457213ff1e3a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:34.537973 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd38739-e1c9-43de-afa4-457213ff1e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:34.556286 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-abd38739-e1c9-43de-afa4-457213ff1e3a None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:34.557819 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-abd38739-e1c9-43de-afa4-457213ff1e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:34.610747 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:34.624689 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 576/1148] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:56:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:56:34.647493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d993e9-6d90-49b3-84e0-ddaee3fb907b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:34.747948 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d993e9-6d90-49b3-84e0-ddaee3fb907b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:34.771032 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:34.785826 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74c8cc-4b77-4045-bf3d-b49ded739b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:34.785826 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74c8cc-4b77-4045-bf3d-b49ded739b4d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:34.785826 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74c8cc-4b77-4045-bf3d-b49ded739b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:34.785826 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74c8cc-4b77-4045-bf3d-b49ded739b4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:34.793205 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74c8cc-4b77-4045-bf3d-b49ded739b4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:56:34.797837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-7b74c8cc-4b77-4045-bf3d-b49ded739b4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:56:34.797837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:56:34.797837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:56:34.797837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:56:34.797837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:56:34.797837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:56:34.797837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:56:34.797837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:56:34.797837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:56:34.797837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:56:34.797837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:56:34.797837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:56:34.797837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:56:34.797837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:56:34.797837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:56:34.797837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:56:34.798929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:56:34.798929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:56:34.798929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:56:34.798929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:56:34.798929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:56:34.798929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:56:34.798929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:56:34.798929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:56:34.798929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:56:34.798929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:56:34.798929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:56:34.798929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:56:34.798929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:56:34.798929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:56:34.798929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:56:34.798929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:56:34.799801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:56:34.799801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:56:34.799801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:56:34.799801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:56:34.799801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:56:34.799801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:56:34.799801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:56:34.799801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:56:34.799801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:56:34.799801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:56:34.799801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:56:34.799801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:56:34.799801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 573/1149] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:56:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:56:34.805248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd38739-e1c9-43de-afa4-457213ff1e3a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:34.905138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd38739-e1c9-43de-afa4-457213ff1e3a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:34.938696 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f45f299-b816-47f9-93f4-1cff1b214505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:34.939294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45f299-b816-47f9-93f4-1cff1b214505 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:34.939566 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45f299-b816-47f9-93f4-1cff1b214505 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:34.939831 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45f299-b816-47f9-93f4-1cff1b214505 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:34.940483 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f45f299-b816-47f9-93f4-1cff1b214505 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:34.940780 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f45f299-b816-47f9-93f4-1cff1b214505 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:34.942178 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f45f299-b816-47f9-93f4-1cff1b214505 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:34.950909 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 577/1150] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:56:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:34.957008 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74c8cc-4b77-4045-bf3d-b49ded739b4d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:35.053765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74c8cc-4b77-4045-bf3d-b49ded739b4d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:35.088420 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e809a8-5b80-489a-aa55-97cd1b9336a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:35.089036 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e809a8-5b80-489a-aa55-97cd1b9336a0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:35.089309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e809a8-5b80-489a-aa55-97cd1b9336a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:35.089575 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e809a8-5b80-489a-aa55-97cd1b9336a0 None admin] PATH_INFO: `/v3/users/01d00ebfebb84c5d96b98060c3784788` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:35.090732 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e809a8-5b80-489a-aa55-97cd1b9336a0 None admin] RBAC: Authorizing `identity:get_user(user_id=01d00ebfebb84c5d96b98060c3784788)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:35.092154 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e809a8-5b80-489a-aa55-97cd1b9336a0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:35.093737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 574/1151] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:56:34 2022] GET /identity/v3/users/01d00ebfebb84c5d96b98060c3784788 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:35.099591 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f45f299-b816-47f9-93f4-1cff1b214505 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:35.151582 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:35.201317 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f45f299-b816-47f9-93f4-1cff1b214505 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:35.234678 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eebc675-f548-45c3-96de-f05565852e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:35.235302 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebc675-f548-45c3-96de-f05565852e01 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:35.235586 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebc675-f548-45c3-96de-f05565852e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:35.235855 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebc675-f548-45c3-96de-f05565852e01 None admin] PATH_INFO: `/v3/projects/ff63ef613ad34164a1bb27a0b3f0ef29` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:35.236963 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eebc675-f548-45c3-96de-f05565852e01 None admin] RBAC: Authorizing `identity:get_project(project_id=ff63ef613ad34164a1bb27a0b3f0ef29)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:35.238415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eebc675-f548-45c3-96de-f05565852e01 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:35.240049 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 578/1152] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:56:35 2022] GET /identity/v3/projects/ff63ef613ad34164a1bb27a0b3f0ef29 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:35.246648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e809a8-5b80-489a-aa55-97cd1b9336a0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:35.298339 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:35.345772 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e809a8-5b80-489a-aa55-97cd1b9336a0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:35.379311 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212ffd5-755b-4ab1-a2f7-7f19fcaca7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:35.379961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212ffd5-755b-4ab1-a2f7-7f19fcaca7e4 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:35.380237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212ffd5-755b-4ab1-a2f7-7f19fcaca7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:35.380507 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212ffd5-755b-4ab1-a2f7-7f19fcaca7e4 None admin] PATH_INFO: `/v3/projects/ff63ef613ad34164a1bb27a0b3f0ef29/users/01d00ebfebb84c5d96b98060c3784788/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:35.382515 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212ffd5-755b-4ab1-a2f7-7f19fcaca7e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff63ef613ad34164a1bb27a0b3f0ef29, user_id=01d00ebfebb84c5d96b98060c3784788, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:35.386326 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212ffd5-755b-4ab1-a2f7-7f19fcaca7e4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:35.474760 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 575/1153] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 10 16:56:35 2022] PUT /identity/v3/projects/ff63ef613ad34164a1bb27a0b3f0ef29/users/01d00ebfebb84c5d96b98060c3784788/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:37.483209 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6aeec5-159b-4ff4-bac9-f1750256b647 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:37.483778 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6aeec5-159b-4ff4-bac9-f1750256b647 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:37.483778 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6aeec5-159b-4ff4-bac9-f1750256b647 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:37.485091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 579/1154] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:56:37.490434 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68f188-5f9f-474c-b86d-728a30ab5472 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:37.490849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68f188-5f9f-474c-b86d-728a30ab5472 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:37.490957 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68f188-5f9f-474c-b86d-728a30ab5472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:37.508966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6f68f188-5f9f-474c-b86d-728a30ab5472 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:37.510385 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6f68f188-5f9f-474c-b86d-728a30ab5472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:37.577523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 576/1155] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:56:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:56:37.600806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6aeec5-159b-4ff4-bac9-f1750256b647 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:37.710513 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6aeec5-159b-4ff4-bac9-f1750256b647 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:37.729581 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:37.748108 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850cb8ba-aefe-43ea-9020-f5bd715ed970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:37.748690 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cb8ba-aefe-43ea-9020-f5bd715ed970 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:37.748962 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cb8ba-aefe-43ea-9020-f5bd715ed970 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:37.749225 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cb8ba-aefe-43ea-9020-f5bd715ed970 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:37.760401 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850cb8ba-aefe-43ea-9020-f5bd715ed970 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:56:37.765425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-850cb8ba-aefe-43ea-9020-f5bd715ed970 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:56:37.765425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:56:37.765425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:56:37.765425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:56:37.765425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:56:37.765425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:56:37.765425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:56:37.765425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:56:37.765425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:56:37.765425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:56:37.765425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:56:37.765425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:56:37.765425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:56:37.765425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:56:37.765425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:56:37.765425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:56:37.766572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:56:37.766572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:56:37.766572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:56:37.766572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:56:37.766572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:56:37.766572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:56:37.766572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:56:37.766572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:56:37.766572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:56:37.766572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:56:37.766572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:56:37.766572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:56:37.766572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:56:37.766572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:56:37.766572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:56:37.766572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:56:37.767654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:56:37.767654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:56:37.767654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:56:37.767654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:56:37.767654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:56:37.767654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:56:37.767654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:56:37.767654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:56:37.767654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:56:37.767654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:56:37.767654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:56:37.767654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:56:37.767654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 580/1156] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:56:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:56:37.772925 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68f188-5f9f-474c-b86d-728a30ab5472 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:37.816776 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:37.872977 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68f188-5f9f-474c-b86d-728a30ab5472 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:37.907486 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7948db5-d7ef-4ae1-b229-1ae77853e8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:37.908107 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7948db5-d7ef-4ae1-b229-1ae77853e8ba None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:37.908364 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7948db5-d7ef-4ae1-b229-1ae77853e8ba None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:37.908631 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7948db5-d7ef-4ae1-b229-1ae77853e8ba None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:37.909296 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7948db5-d7ef-4ae1-b229-1ae77853e8ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:37.909595 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7948db5-d7ef-4ae1-b229-1ae77853e8ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:37.910998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7948db5-d7ef-4ae1-b229-1ae77853e8ba None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:37.919969 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 577/1157] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:56:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:37.925938 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850cb8ba-aefe-43ea-9020-f5bd715ed970 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:38.026550 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850cb8ba-aefe-43ea-9020-f5bd715ed970 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:38.058931 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28400733-9973-4607-82ef-eb4283121bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:38.059547 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28400733-9973-4607-82ef-eb4283121bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:38.059826 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28400733-9973-4607-82ef-eb4283121bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:38.060098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28400733-9973-4607-82ef-eb4283121bd1 None admin] PATH_INFO: `/v3/users/01d00ebfebb84c5d96b98060c3784788` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:38.061203 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28400733-9973-4607-82ef-eb4283121bd1 None admin] RBAC: Authorizing `identity:get_user(user_id=01d00ebfebb84c5d96b98060c3784788)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:38.062714 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28400733-9973-4607-82ef-eb4283121bd1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:38.064282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 581/1158] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:56:37 2022] GET /identity/v3/users/01d00ebfebb84c5d96b98060c3784788 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:38.070089 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7948db5-d7ef-4ae1-b229-1ae77853e8ba None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:38.167914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7948db5-d7ef-4ae1-b229-1ae77853e8ba None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:38.170165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:38.200695 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b5c6b-270b-49b2-bf22-e76410bdf435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:38.201282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b5c6b-270b-49b2-bf22-e76410bdf435 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:38.201616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b5c6b-270b-49b2-bf22-e76410bdf435 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:38.202048 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b5c6b-270b-49b2-bf22-e76410bdf435 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:38.267858 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b5c6b-270b-49b2-bf22-e76410bdf435 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:38.269291 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b5c6b-270b-49b2-bf22-e76410bdf435 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:38.364816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 578/1159] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:56:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:56:38.370897 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28400733-9973-4607-82ef-eb4283121bd1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:38.416568 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:38.476017 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28400733-9973-4607-82ef-eb4283121bd1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:38.508249 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95105adc-27d4-4858-a998-db47b11bfd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:38.508875 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95105adc-27d4-4858-a998-db47b11bfd6b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:38.509673 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95105adc-27d4-4858-a998-db47b11bfd6b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:38.509938 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95105adc-27d4-4858-a998-db47b11bfd6b None admin] PATH_INFO: `/v3/projects/ff63ef613ad34164a1bb27a0b3f0ef29` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:38.511073 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95105adc-27d4-4858-a998-db47b11bfd6b None admin] RBAC: Authorizing `identity:get_project(project_id=ff63ef613ad34164a1bb27a0b3f0ef29)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:38.512495 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95105adc-27d4-4858-a998-db47b11bfd6b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:38.514117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 582/1160] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:56:38 2022] GET /identity/v3/projects/ff63ef613ad34164a1bb27a0b3f0ef29 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:38.520336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b5c6b-270b-49b2-bf22-e76410bdf435 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:38.523611 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:38.620927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b5c6b-270b-49b2-bf22-e76410bdf435 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:38.655223 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f874703-8e5d-4097-ad65-7d5a9130143c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:38.655826 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f874703-8e5d-4097-ad65-7d5a9130143c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:38.656102 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f874703-8e5d-4097-ad65-7d5a9130143c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:38.656371 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f874703-8e5d-4097-ad65-7d5a9130143c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:38.657092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f874703-8e5d-4097-ad65-7d5a9130143c None admin] RBAC: Adding query filter params (role.id=7554b353873942fab506de830188aba0, scope.project.id=ff63ef613ad34164a1bb27a0b3f0ef29, user.id=01d00ebfebb84c5d96b98060c3784788) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:38.657394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f874703-8e5d-4097-ad65-7d5a9130143c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:38.658854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f874703-8e5d-4097-ad65-7d5a9130143c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:38.664733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 579/1161] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:56:38 2022] GET /identity/v3/role_assignments?user.id=01d00ebfebb84c5d96b98060c3784788&scope.project.id=ff63ef613ad34164a1bb27a0b3f0ef29&role.id=7554b353873942fab506de830188aba0 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:40.702008 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10adecc0-3e1b-42c8-b48b-0a87147fdeb1 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:40.702578 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10adecc0-3e1b-42c8-b48b-0a87147fdeb1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:40.702578 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10adecc0-3e1b-42c8-b48b-0a87147fdeb1 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:40.703907 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 583/1162] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:56:40.709035 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a5e5b-627e-4db8-8531-f73a1f8c8a94 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:40.709442 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a5e5b-627e-4db8-8531-f73a1f8c8a94 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:40.709570 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a5e5b-627e-4db8-8531-f73a1f8c8a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:40.727434 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-fc8a5e5b-627e-4db8-8531-f73a1f8c8a94 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:40.729291 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fc8a5e5b-627e-4db8-8531-f73a1f8c8a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:40.789999 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:40.796126 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 580/1163] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:56:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:56:40.818903 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10adecc0-3e1b-42c8-b48b-0a87147fdeb1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:40.921314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10adecc0-3e1b-42c8-b48b-0a87147fdeb1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:40.934803 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:40.953912 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e136fa-5141-4ec8-ac87-d74c57a8d6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:40.954523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e136fa-5141-4ec8-ac87-d74c57a8d6fa None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:40.954795 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e136fa-5141-4ec8-ac87-d74c57a8d6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:40.955058 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e136fa-5141-4ec8-ac87-d74c57a8d6fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:41.018496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e136fa-5141-4ec8-ac87-d74c57a8d6fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:41.019921 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e136fa-5141-4ec8-ac87-d74c57a8d6fa None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:41.113981 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 584/1164] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:56:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 16:56:41.120081 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a5e5b-627e-4db8-8531-f73a1f8c8a94 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:41.214804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a5e5b-627e-4db8-8531-f73a1f8c8a94 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:41.247725 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f8e15-77d2-436e-a4c3-4966c77b4e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:41.248324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f8e15-77d2-436e-a4c3-4966c77b4e90 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:41.248592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f8e15-77d2-436e-a4c3-4966c77b4e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:41.248856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f8e15-77d2-436e-a4c3-4966c77b4e90 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:41.255212 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78f8e15-77d2-436e-a4c3-4966c77b4e90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:41.256623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78f8e15-77d2-436e-a4c3-4966c77b4e90 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:41.267862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-c78f8e15-77d2-436e-a4c3-4966c77b4e90 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 10 16:56:41.267862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:56:41.267862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 10 16:56:41.267862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 10 16:56:41.267862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:56:41.267862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:56:41.267862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:56:41.267862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:56:41.267862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 10 16:56:41.267862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:56:41.267862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:56:41.267862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:56:41.267862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:56:41.267862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:56:41.267862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:56:41.267862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:56:41.267862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:56:41.269216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:56:41.269216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:56:41.269216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:56:41.269216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:56:41.269216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:56:41.269216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:56:41.269216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:56:41.269216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:56:41.269216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:56:41.269216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:56:41.269216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:56:41.269216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:56:41.269216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:56:41.269216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:56:41.269216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:56:41.269216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:56:41.270528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:56:41.270528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:56:41.270528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:56:41.270528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:56:41.270528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:56:41.270528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:56:41.270528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:56:41.270528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:56:41.270528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:56:41.270528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:56:41.270528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 10 16:56:41.270528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:56:41.270528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 10 16:56:41.270528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:56:41.270528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 581/1165] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 10 16:56:41 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:56:41.275772 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e136fa-5141-4ec8-ac87-d74c57a8d6fa None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:41.290304 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:41.372770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e136fa-5141-4ec8-ac87-d74c57a8d6fa None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:41.410959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba19aaf-0bc8-48bd-8dce-fab8fa5777ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:41.411565 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba19aaf-0bc8-48bd-8dce-fab8fa5777ce None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:41.411841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba19aaf-0bc8-48bd-8dce-fab8fa5777ce None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:41.412107 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba19aaf-0bc8-48bd-8dce-fab8fa5777ce None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:41.412743 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba19aaf-0bc8-48bd-8dce-fab8fa5777ce None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:41.413042 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba19aaf-0bc8-48bd-8dce-fab8fa5777ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:41.414430 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba19aaf-0bc8-48bd-8dce-fab8fa5777ce None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:41.422011 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 585/1166] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Aug 10 16:56:41 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:43.409473 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f0275-5a22-4b34-ae9a-e63d1db84c81 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:43.410033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f0275-5a22-4b34-ae9a-e63d1db84c81 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:43.410033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f0275-5a22-4b34-ae9a-e63d1db84c81 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:43.411474 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 582/1167] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:56:43.416918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51f1be-63ad-43d7-b967-78c554c0f54e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:43.417336 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51f1be-63ad-43d7-b967-78c554c0f54e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:43.417440 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51f1be-63ad-43d7-b967-78c554c0f54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:43.435499 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ec51f1be-63ad-43d7-b967-78c554c0f54e None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:43.436721 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ec51f1be-63ad-43d7-b967-78c554c0f54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:43.501433 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 586/1168] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:56:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:56:43.524170 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f0275-5a22-4b34-ae9a-e63d1db84c81 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:43.581150 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:43.622951 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f0275-5a22-4b34-ae9a-e63d1db84c81 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:43.656150 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639abaa-7c88-4f57-ab5a-402a02cbb947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:43.656743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639abaa-7c88-4f57-ab5a-402a02cbb947 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:43.657019 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639abaa-7c88-4f57-ab5a-402a02cbb947 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:43.657289 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639abaa-7c88-4f57-ab5a-402a02cbb947 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:43.657948 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639abaa-7c88-4f57-ab5a-402a02cbb947 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:43.659338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639abaa-7c88-4f57-ab5a-402a02cbb947 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:43.747480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 583/1169] 10.222.0.45 () {60 vars in 1138 bytes} [Wed Aug 10 16:56:43 2022] POST /identity/v3/domains => generated 267 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:56:45.771310 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49967b-af1d-4383-9655-b8d4c0213a88 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:45.771831 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49967b-af1d-4383-9655-b8d4c0213a88 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:45.771957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49967b-af1d-4383-9655-b8d4c0213a88 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:45.773466 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 587/1170] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:56:45.779579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2b8b4-6ace-4235-9ca8-83a95c95958c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:45.780005 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2b8b4-6ace-4235-9ca8-83a95c95958c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:45.780112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2b8b4-6ace-4235-9ca8-83a95c95958c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:45.798631 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-72e2b8b4-6ace-4235-9ca8-83a95c95958c None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:45.799833 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-72e2b8b4-6ace-4235-9ca8-83a95c95958c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:45.867742 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 584/1171] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:56:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:56:45.892697 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49967b-af1d-4383-9655-b8d4c0213a88 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:45.904808 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:45.999800 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49967b-af1d-4383-9655-b8d4c0213a88 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:46.036726 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ef5ea-f84b-4f0a-968d-c28b2442b057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:46.037402 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ef5ea-f84b-4f0a-968d-c28b2442b057 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:46.037720 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ef5ea-f84b-4f0a-968d-c28b2442b057 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:46.038049 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ef5ea-f84b-4f0a-968d-c28b2442b057 None admin] PATH_INFO: `/v3/domains/0e5738dd1d6646e8857ca4bb74a34a05` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:46.048614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ef5ea-f84b-4f0a-968d-c28b2442b057 None admin] RBAC: Authorizing `identity:get_domain(domain_id=0e5738dd1d6646e8857ca4bb74a34a05)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:46.050057 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ef5ea-f84b-4f0a-968d-c28b2442b057 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:46.051739 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 588/1172] 10.222.0.45 () {56 vars in 1183 bytes} [Wed Aug 10 16:56:45 2022] GET /identity/v3/domains/0e5738dd1d6646e8857ca4bb74a34a05 => generated 267 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:46.057888 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2b8b4-6ace-4235-9ca8-83a95c95958c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:46.106735 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:46.156991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2b8b4-6ace-4235-9ca8-83a95c95958c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:46.190970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d64461-7e64-4543-8a3d-2288e85e1a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:46.191563 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d64461-7e64-4543-8a3d-2288e85e1a62 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:46.191834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d64461-7e64-4543-8a3d-2288e85e1a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:46.192095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d64461-7e64-4543-8a3d-2288e85e1a62 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:46.192731 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d64461-7e64-4543-8a3d-2288e85e1a62 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:46.194215 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d64461-7e64-4543-8a3d-2288e85e1a62 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:46.292993 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 585/1173] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 10 16:56:46 2022] POST /identity/v3/projects => generated 361 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:56:48.352283 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cca1c3-ce7d-4648-b645-69d699528ba2 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:48.353397 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cca1c3-ce7d-4648-b645-69d699528ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:48.353511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cca1c3-ce7d-4648-b645-69d699528ba2 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:48.354904 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 589/1174] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:56:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:56:48.360766 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587d24f-8016-4413-82b5-03be5d0ec5c4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:48.361202 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587d24f-8016-4413-82b5-03be5d0ec5c4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:48.361331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587d24f-8016-4413-82b5-03be5d0ec5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:48.379283 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3587d24f-8016-4413-82b5-03be5d0ec5c4 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:48.380469 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3587d24f-8016-4413-82b5-03be5d0ec5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:48.448577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 586/1175] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:56:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:56:48.473605 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cca1c3-ce7d-4648-b645-69d699528ba2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:48.543354 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:48.577792 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cca1c3-ce7d-4648-b645-69d699528ba2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:48.615821 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305dbf72-0688-4654-b565-484466924742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:48.616455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dbf72-0688-4654-b565-484466924742 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:48.616739 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dbf72-0688-4654-b565-484466924742 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:48.617012 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dbf72-0688-4654-b565-484466924742 None admin] PATH_INFO: `/v3/domains/0e5738dd1d6646e8857ca4bb74a34a05` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:48.618137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305dbf72-0688-4654-b565-484466924742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=0e5738dd1d6646e8857ca4bb74a34a05)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:48.619821 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305dbf72-0688-4654-b565-484466924742 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:48.621401 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 590/1176] 10.222.0.45 () {56 vars in 1183 bytes} [Wed Aug 10 16:56:48 2022] GET /identity/v3/domains/0e5738dd1d6646e8857ca4bb74a34a05 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:48.627186 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587d24f-8016-4413-82b5-03be5d0ec5c4 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:48.656398 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:48.725800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587d24f-8016-4413-82b5-03be5d0ec5c4 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:48.761156 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06852c-3519-4ba8-b45a-3b10f6c22d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:48.761755 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06852c-3519-4ba8-b45a-3b10f6c22d0f None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:48.762032 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06852c-3519-4ba8-b45a-3b10f6c22d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:48.762299 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06852c-3519-4ba8-b45a-3b10f6c22d0f None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:48.762968 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06852c-3519-4ba8-b45a-3b10f6c22d0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:48.764393 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06852c-3519-4ba8-b45a-3b10f6c22d0f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:48.843245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 587/1177] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 10 16:56:48 2022] POST /identity/v3/users => generated 311 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:56:50.899334 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad323842-f343-4ff7-8166-c90b3c403c22 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:50.900720 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad323842-f343-4ff7-8166-c90b3c403c22 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:50.900720 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad323842-f343-4ff7-8166-c90b3c403c22 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:50.901760 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 591/1178] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:56:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:56:50.907255 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbd94e-cc23-4d8a-bcdc-63111537beca None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:50.907709 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbd94e-cc23-4d8a-bcdc-63111537beca None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:50.907819 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbd94e-cc23-4d8a-bcdc-63111537beca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:50.928651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f5dbd94e-cc23-4d8a-bcdc-63111537beca None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:50.929847 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f5dbd94e-cc23-4d8a-bcdc-63111537beca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:51.002670 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 588/1179] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:56:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:56:51.027395 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad323842-f343-4ff7-8166-c90b3c403c22 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:51.135565 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad323842-f343-4ff7-8166-c90b3c403c22 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:51.172126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89074e04-e86e-4e32-8b6a-f21b524bfd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:51.172667 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89074e04-e86e-4e32-8b6a-f21b524bfd48 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:51.172939 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89074e04-e86e-4e32-8b6a-f21b524bfd48 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:51.173204 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89074e04-e86e-4e32-8b6a-f21b524bfd48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:51.175853 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:51.184758 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89074e04-e86e-4e32-8b6a-f21b524bfd48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:56:51.189419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-89074e04-e86e-4e32-8b6a-f21b524bfd48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:56:51.189419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:56:51.189419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:56:51.189419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:56:51.189419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:56:51.189419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:56:51.189419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:56:51.189419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:56:51.189419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:56:51.189419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:56:51.189419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:56:51.189419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:56:51.189419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:56:51.189419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:56:51.189419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:56:51.189419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:56:51.196768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:56:51.196768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:56:51.196768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:56:51.196768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:56:51.196768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:56:51.196768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:56:51.196768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:56:51.196768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:56:51.196768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:56:51.196768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:56:51.196768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:56:51.196768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:56:51.196768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:56:51.196768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:56:51.196768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:56:51.196768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:56:51.197888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:56:51.197888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:56:51.197888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:56:51.197888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:56:51.197888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:56:51.197888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:56:51.197888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:56:51.197888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:56:51.197888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:56:51.197888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:56:51.197888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:56:51.197888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:56:51.197888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 592/1180] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:56:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:56:51.198847 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbd94e-cc23-4d8a-bcdc-63111537beca None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:51.215133 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:51.302900 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbd94e-cc23-4d8a-bcdc-63111537beca None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:51.336900 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9e0cb-5902-487d-8b3d-241b49380419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:51.338437 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9e0cb-5902-487d-8b3d-241b49380419 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:51.338720 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9e0cb-5902-487d-8b3d-241b49380419 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:51.338993 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9e0cb-5902-487d-8b3d-241b49380419 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:51.339672 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9e0cb-5902-487d-8b3d-241b49380419 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:51.339975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9e0cb-5902-487d-8b3d-241b49380419 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:51.341342 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9e0cb-5902-487d-8b3d-241b49380419 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:51.349749 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 589/1181] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:56:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:51.356168 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89074e04-e86e-4e32-8b6a-f21b524bfd48 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:51.456650 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89074e04-e86e-4e32-8b6a-f21b524bfd48 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:51.489749 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f3fb9-b57b-4714-b558-4a9f6edc510b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:51.490400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f3fb9-b57b-4714-b558-4a9f6edc510b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:51.490685 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f3fb9-b57b-4714-b558-4a9f6edc510b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:51.490957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f3fb9-b57b-4714-b558-4a9f6edc510b None admin] PATH_INFO: `/v3/users/9d5a069aa38c4eebb0d58d4591bc7c6b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:51.504271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f3fb9-b57b-4714-b558-4a9f6edc510b None admin] RBAC: Authorizing `identity:get_user(user_id=9d5a069aa38c4eebb0d58d4591bc7c6b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:51.505723 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f3fb9-b57b-4714-b558-4a9f6edc510b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:51.507403 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 593/1182] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:56:51 2022] GET /identity/v3/users/9d5a069aa38c4eebb0d58d4591bc7c6b => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:51.513947 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9e0cb-5902-487d-8b3d-241b49380419 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:51.605701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:51.628451 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9e0cb-5902-487d-8b3d-241b49380419 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:51.662292 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947408d6-e18a-40b5-a4e4-d9f120716086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:51.662922 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947408d6-e18a-40b5-a4e4-d9f120716086 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:51.663197 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947408d6-e18a-40b5-a4e4-d9f120716086 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:51.663466 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947408d6-e18a-40b5-a4e4-d9f120716086 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:51.730688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947408d6-e18a-40b5-a4e4-d9f120716086 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:51.732086 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947408d6-e18a-40b5-a4e4-d9f120716086 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:51.808961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:51.831176 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 590/1183] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:56:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:56:51.837288 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f3fb9-b57b-4714-b558-4a9f6edc510b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:51.896305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:51.944423 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f3fb9-b57b-4714-b558-4a9f6edc510b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:51.979176 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1957eeb-aad9-497b-9ed9-a1e2608f2de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:51.979799 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1957eeb-aad9-497b-9ed9-a1e2608f2de5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:51.980077 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1957eeb-aad9-497b-9ed9-a1e2608f2de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:51.980348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1957eeb-aad9-497b-9ed9-a1e2608f2de5 None admin] PATH_INFO: `/v3/projects/fa7da358a72a41e4a32c3b7a9b84ece2` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:51.981534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1957eeb-aad9-497b-9ed9-a1e2608f2de5 None admin] RBAC: Authorizing `identity:get_project(project_id=fa7da358a72a41e4a32c3b7a9b84ece2)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:51.982950 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1957eeb-aad9-497b-9ed9-a1e2608f2de5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:51.984584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 594/1184] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:56:51 2022] GET /identity/v3/projects/fa7da358a72a41e4a32c3b7a9b84ece2 => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:51.990837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947408d6-e18a-40b5-a4e4-d9f120716086 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:52.095518 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947408d6-e18a-40b5-a4e4-d9f120716086 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:52.131669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3763ab36-183b-4a1e-9a18-0bafa5780fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:52.132326 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763ab36-183b-4a1e-9a18-0bafa5780fe1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:52.132619 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763ab36-183b-4a1e-9a18-0bafa5780fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:52.132916 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763ab36-183b-4a1e-9a18-0bafa5780fe1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:52.133700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3763ab36-183b-4a1e-9a18-0bafa5780fe1 None admin] RBAC: Adding query filter params (role.id=7554b353873942fab506de830188aba0, scope.project.id=fa7da358a72a41e4a32c3b7a9b84ece2, user.id=9d5a069aa38c4eebb0d58d4591bc7c6b) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:52.134708 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3763ab36-183b-4a1e-9a18-0bafa5780fe1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:52.136491 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3763ab36-183b-4a1e-9a18-0bafa5780fe1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:52.142896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 591/1185] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:56:51 2022] GET /identity/v3/role_assignments?user.id=9d5a069aa38c4eebb0d58d4591bc7c6b&scope.project.id=fa7da358a72a41e4a32c3b7a9b84ece2&role.id=7554b353873942fab506de830188aba0 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:54.154305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4b150-7cff-4d51-b004-5ee4a2549e96 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:54.154891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4b150-7cff-4d51-b004-5ee4a2549e96 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:54.154891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4b150-7cff-4d51-b004-5ee4a2549e96 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:54.156211 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 595/1186] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:56:54.161331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969c8a0-8ac4-4f34-9de0-fcf9f91db84a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:54.161704 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969c8a0-8ac4-4f34-9de0-fcf9f91db84a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:54.161864 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969c8a0-8ac4-4f34-9de0-fcf9f91db84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:54.179317 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3969c8a0-8ac4-4f34-9de0-fcf9f91db84a None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:54.180484 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3969c8a0-8ac4-4f34-9de0-fcf9f91db84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:54.214984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:54.244830 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 592/1187] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:56:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:56:54.267508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4b150-7cff-4d51-b004-5ee4a2549e96 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:54.371887 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4b150-7cff-4d51-b004-5ee4a2549e96 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:54.393515 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:54.405337 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bebd5ef-50fd-4da4-bcaa-b041889de03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:54.405875 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebd5ef-50fd-4da4-bcaa-b041889de03b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:54.406148 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebd5ef-50fd-4da4-bcaa-b041889de03b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:54.406439 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebd5ef-50fd-4da4-bcaa-b041889de03b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:54.416936 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bebd5ef-50fd-4da4-bcaa-b041889de03b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:56:54.421893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5bebd5ef-50fd-4da4-bcaa-b041889de03b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:56:54.421893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:56:54.421893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:56:54.421893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:56:54.421893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:56:54.421893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:56:54.421893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:56:54.421893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:56:54.421893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:56:54.421893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:56:54.421893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:56:54.421893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:56:54.421893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:56:54.421893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:56:54.421893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:56:54.421893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:56:54.422860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:56:54.422860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:56:54.422860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:56:54.422860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:56:54.422860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:56:54.422860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:56:54.422860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:56:54.422860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:56:54.422860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:56:54.422860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:56:54.422860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:56:54.422860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:56:54.422860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:56:54.422860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:56:54.422860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:56:54.422860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:56:54.423770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:56:54.423770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:56:54.423770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:56:54.423770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:56:54.423770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:56:54.423770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:56:54.423770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:56:54.423770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:56:54.423770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:56:54.423770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:56:54.423770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:56:54.423770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:56:54.423770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 596/1188] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:56:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:56:54.429099 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3969c8a0-8ac4-4f34-9de0-fcf9f91db84a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:54.527294 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3969c8a0-8ac4-4f34-9de0-fcf9f91db84a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:54.562416 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b3010-8caf-4b35-ba3c-65968e3adbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:54.563015 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b3010-8caf-4b35-ba3c-65968e3adbce None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:54.563290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b3010-8caf-4b35-ba3c-65968e3adbce None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:54.563560 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b3010-8caf-4b35-ba3c-65968e3adbce None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:54.564229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b3010-8caf-4b35-ba3c-65968e3adbce None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:54.564529 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b3010-8caf-4b35-ba3c-65968e3adbce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:54.565892 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b3010-8caf-4b35-ba3c-65968e3adbce None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:54.574631 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 593/1189] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:56:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:54.580574 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bebd5ef-50fd-4da4-bcaa-b041889de03b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:54.684801 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bebd5ef-50fd-4da4-bcaa-b041889de03b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:54.720079 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67229b5-4e55-4645-86c5-9829c05009fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:54.720702 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67229b5-4e55-4645-86c5-9829c05009fd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:54.720977 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67229b5-4e55-4645-86c5-9829c05009fd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:54.721246 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67229b5-4e55-4645-86c5-9829c05009fd None admin] PATH_INFO: `/v3/users/9d5a069aa38c4eebb0d58d4591bc7c6b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:54.722382 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67229b5-4e55-4645-86c5-9829c05009fd None admin] RBAC: Authorizing `identity:get_user(user_id=9d5a069aa38c4eebb0d58d4591bc7c6b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:54.723812 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67229b5-4e55-4645-86c5-9829c05009fd None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:54.725372 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 597/1190] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:56:54 2022] GET /identity/v3/users/9d5a069aa38c4eebb0d58d4591bc7c6b => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:54.731926 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b3010-8caf-4b35-ba3c-65968e3adbce None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:54.772905 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:54.839738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b3010-8caf-4b35-ba3c-65968e3adbce None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:54.874821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9baf856-24f0-46a3-be39-0c2b6fd9c7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:54.875439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9baf856-24f0-46a3-be39-0c2b6fd9c7cb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:54.875716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9baf856-24f0-46a3-be39-0c2b6fd9c7cb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:54.876431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9baf856-24f0-46a3-be39-0c2b6fd9c7cb None admin] PATH_INFO: `/v3/projects/fa7da358a72a41e4a32c3b7a9b84ece2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:54.877241 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9baf856-24f0-46a3-be39-0c2b6fd9c7cb None admin] RBAC: Authorizing `identity:get_project(project_id=fa7da358a72a41e4a32c3b7a9b84ece2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:54.878816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9baf856-24f0-46a3-be39-0c2b6fd9c7cb None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:54.880484 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 594/1191] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:56:54 2022] GET /identity/v3/projects/fa7da358a72a41e4a32c3b7a9b84ece2 => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:54.886603 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67229b5-4e55-4645-86c5-9829c05009fd None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:54.937782 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:54.985421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67229b5-4e55-4645-86c5-9829c05009fd None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:55.019602 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d33a1-56f5-47d6-ab52-0db9c5c1b8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:55.020259 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d33a1-56f5-47d6-ab52-0db9c5c1b8a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:55.020536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d33a1-56f5-47d6-ab52-0db9c5c1b8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:55.020806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d33a1-56f5-47d6-ab52-0db9c5c1b8a9 None admin] PATH_INFO: `/v3/projects/fa7da358a72a41e4a32c3b7a9b84ece2/users/9d5a069aa38c4eebb0d58d4591bc7c6b/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:55.022722 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d33a1-56f5-47d6-ab52-0db9c5c1b8a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=fa7da358a72a41e4a32c3b7a9b84ece2, user_id=9d5a069aa38c4eebb0d58d4591bc7c6b, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:55.024518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d33a1-56f5-47d6-ab52-0db9c5c1b8a9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:55.128782 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 598/1192] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 10 16:56:54 2022] PUT /identity/v3/projects/fa7da358a72a41e4a32c3b7a9b84ece2/users/9d5a069aa38c4eebb0d58d4591bc7c6b/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:56:57.169727 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e71bc-8771-41bf-a619-27751bd5a70c None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:57.170301 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e71bc-8771-41bf-a619-27751bd5a70c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:57.170301 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e71bc-8771-41bf-a619-27751bd5a70c None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:57.171624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 595/1193] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:56:57.177025 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a994fef-3d75-49ef-97d8-e673553de23d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:57.177492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a994fef-3d75-49ef-97d8-e673553de23d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:57.177605 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a994fef-3d75-49ef-97d8-e673553de23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:57.195531 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5a994fef-3d75-49ef-97d8-e673553de23d None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:56:57.197543 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5a994fef-3d75-49ef-97d8-e673553de23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:56:57.264727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 599/1194] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:56:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:56:57.288926 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e71bc-8771-41bf-a619-27751bd5a70c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:57.394258 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e71bc-8771-41bf-a619-27751bd5a70c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:57.395841 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:57.429865 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83291f48-0b24-479b-a74c-9dfdba211e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:57.430393 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83291f48-0b24-479b-a74c-9dfdba211e3f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:57.431496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83291f48-0b24-479b-a74c-9dfdba211e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:57.431788 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83291f48-0b24-479b-a74c-9dfdba211e3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:57.442527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83291f48-0b24-479b-a74c-9dfdba211e3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:56:57.447793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-83291f48-0b24-479b-a74c-9dfdba211e3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:56:57.447793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:56:57.447793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:56:57.447793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:56:57.447793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:56:57.447793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:56:57.447793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:56:57.447793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:56:57.447793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:56:57.447793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:56:57.447793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:56:57.447793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:56:57.447793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:56:57.447793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:56:57.447793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:56:57.447793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:56:57.448949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:56:57.448949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:56:57.448949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:56:57.448949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:56:57.448949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:56:57.448949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:56:57.448949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:56:57.448949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:56:57.448949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:56:57.448949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:56:57.448949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:56:57.448949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:56:57.448949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:56:57.448949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:56:57.448949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:56:57.448949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:56:57.449813 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:56:57.449813 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:56:57.449813 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:56:57.449813 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:56:57.449813 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:56:57.449813 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:56:57.449813 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:56:57.449813 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:56:57.449813 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:56:57.449813 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:56:57.449813 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:56:57.449813 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:56:57.449813 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 596/1195] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:56:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:56:57.455193 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a994fef-3d75-49ef-97d8-e673553de23d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:57.496914 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:57.556267 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a994fef-3d75-49ef-97d8-e673553de23d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:57.596909 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d94a0-2664-4d76-986b-fa379bdd11d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:57.597530 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d94a0-2664-4d76-986b-fa379bdd11d1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:57.597804 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d94a0-2664-4d76-986b-fa379bdd11d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:57.598070 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d94a0-2664-4d76-986b-fa379bdd11d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:57.598782 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d94a0-2664-4d76-986b-fa379bdd11d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:57.599080 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d94a0-2664-4d76-986b-fa379bdd11d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:57.600467 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d94a0-2664-4d76-986b-fa379bdd11d1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:57.610204 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 600/1196] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:56:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:56:57.616670 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83291f48-0b24-479b-a74c-9dfdba211e3f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:57.724382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83291f48-0b24-479b-a74c-9dfdba211e3f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:57.756878 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa522cc-ef95-4f25-b97e-d331e5cc176c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:57.757492 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa522cc-ef95-4f25-b97e-d331e5cc176c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:57.757768 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa522cc-ef95-4f25-b97e-d331e5cc176c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:57.758034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa522cc-ef95-4f25-b97e-d331e5cc176c None admin] PATH_INFO: `/v3/users/9d5a069aa38c4eebb0d58d4591bc7c6b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:57.759713 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa522cc-ef95-4f25-b97e-d331e5cc176c None admin] RBAC: Authorizing `identity:get_user(user_id=9d5a069aa38c4eebb0d58d4591bc7c6b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:57.761134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa522cc-ef95-4f25-b97e-d331e5cc176c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:57.762767 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 597/1197] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 10 16:56:57 2022] GET /identity/v3/users/9d5a069aa38c4eebb0d58d4591bc7c6b => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:57.768722 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d94a0-2664-4d76-986b-fa379bdd11d1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:57.871069 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d94a0-2664-4d76-986b-fa379bdd11d1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:57.872688 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:57.909133 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f554eb4-263f-4667-87e1-892c3cd27c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:57.909730 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f554eb4-263f-4667-87e1-892c3cd27c52 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:57.910008 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f554eb4-263f-4667-87e1-892c3cd27c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:57.910280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f554eb4-263f-4667-87e1-892c3cd27c52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:57.978385 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f554eb4-263f-4667-87e1-892c3cd27c52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:57.979763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f554eb4-263f-4667-87e1-892c3cd27c52 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:58.079368 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 601/1198] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:56:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:56:58.086787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa522cc-ef95-4f25-b97e-d331e5cc176c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:58.118027 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:58.194737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa522cc-ef95-4f25-b97e-d331e5cc176c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:58.229667 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e88dc8-00e0-4ada-99e8-69a074b3006d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:58.230199 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e88dc8-00e0-4ada-99e8-69a074b3006d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:58.230508 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e88dc8-00e0-4ada-99e8-69a074b3006d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:58.230782 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e88dc8-00e0-4ada-99e8-69a074b3006d None admin] PATH_INFO: `/v3/projects/fa7da358a72a41e4a32c3b7a9b84ece2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:58.231899 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e88dc8-00e0-4ada-99e8-69a074b3006d None admin] RBAC: Authorizing `identity:get_project(project_id=fa7da358a72a41e4a32c3b7a9b84ece2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:58.233327 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e88dc8-00e0-4ada-99e8-69a074b3006d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:58.235010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 598/1199] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 10 16:56:58 2022] GET /identity/v3/projects/fa7da358a72a41e4a32c3b7a9b84ece2 => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:56:58.241293 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f554eb4-263f-4667-87e1-892c3cd27c52 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:56:58.244127 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:56:58.343962 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f554eb4-263f-4667-87e1-892c3cd27c52 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:56:58.379198 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb64350-f2f5-4f3a-8376-613062cf757e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:56:58.379802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb64350-f2f5-4f3a-8376-613062cf757e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:56:58.380080 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb64350-f2f5-4f3a-8376-613062cf757e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:56:58.380350 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb64350-f2f5-4f3a-8376-613062cf757e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:56:58.381067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb64350-f2f5-4f3a-8376-613062cf757e None admin] RBAC: Adding query filter params (role.id=7554b353873942fab506de830188aba0, scope.project.id=fa7da358a72a41e4a32c3b7a9b84ece2, user.id=9d5a069aa38c4eebb0d58d4591bc7c6b) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:56:58.381371 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb64350-f2f5-4f3a-8376-613062cf757e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:56:58.382842 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb64350-f2f5-4f3a-8376-613062cf757e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:56:58.389428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 602/1200] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:56:58 2022] GET /identity/v3/role_assignments?user.id=9d5a069aa38c4eebb0d58d4591bc7c6b&scope.project.id=fa7da358a72a41e4a32c3b7a9b84ece2&role.id=7554b353873942fab506de830188aba0 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:57:08.225047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=51 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:57:08.386766 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=67 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:15.757981 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af0eca-f53d-4af3-a672-0cbdd84b80fb None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:15.759261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af0eca-f53d-4af3-a672-0cbdd84b80fb None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:15.759261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af0eca-f53d-4af3-a672-0cbdd84b80fb None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:15.760615 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 599/1201] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:58:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:58:15.765742 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bbaea-81bc-47cb-936c-9e41ddf284fb None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:15.766182 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bbaea-81bc-47cb-936c-9e41ddf284fb None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:15.766283 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bbaea-81bc-47cb-936c-9e41ddf284fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:15.784639 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ff2bbaea-81bc-47cb-936c-9e41ddf284fb None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:58:15.785806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ff2bbaea-81bc-47cb-936c-9e41ddf284fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:15.853661 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 603/1202] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:58:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:58:15.878686 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af0eca-f53d-4af3-a672-0cbdd84b80fb None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:15.980885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af0eca-f53d-4af3-a672-0cbdd84b80fb None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:16.014359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f2067-5978-428c-b6a6-c8cfe4b4f7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:16.014992 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f2067-5978-428c-b6a6-c8cfe4b4f7ad None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:16.015261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f2067-5978-428c-b6a6-c8cfe4b4f7ad None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:16.015523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f2067-5978-428c-b6a6-c8cfe4b4f7ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:16.025773 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372f2067-5978-428c-b6a6-c8cfe4b4f7ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:16.027241 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372f2067-5978-428c-b6a6-c8cfe4b4f7ad None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:16.041409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-372f2067-5978-428c-b6a6-c8cfe4b4f7ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:58:16.041409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:16.041409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:16.041409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:16.041409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:16.041409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:16.041409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:16.041409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:16.041409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:16.041409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:16.041409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:16.041409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:16.041409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:58:16.041409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:58:16.041409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:58:16.041409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:58:16.042624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:16.042624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:16.042624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:16.042624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:16.042624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:16.042624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:16.042624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:16.042624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:16.042624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:16.042624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:16.042624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:16.042624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:16.042624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:16.042624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:16.042624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:16.042624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:16.042624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:58:16.043658 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:58:16.043658 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:58:16.043658 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:58:16.043658 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 600/1203] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:58:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:16.048541 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2bbaea-81bc-47cb-936c-9e41ddf284fb None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:16.148793 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2bbaea-81bc-47cb-936c-9e41ddf284fb None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:16.178189 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:16.184742 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059e78d-99b3-4911-8841-ad290f13cf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:16.185349 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059e78d-99b3-4911-8841-ad290f13cf89 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:16.185624 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059e78d-99b3-4911-8841-ad290f13cf89 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:16.185891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059e78d-99b3-4911-8841-ad290f13cf89 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:16.186559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059e78d-99b3-4911-8841-ad290f13cf89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:16.186860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059e78d-99b3-4911-8841-ad290f13cf89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:16.188482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059e78d-99b3-4911-8841-ad290f13cf89 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:16.199721 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 604/1204] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:16.205508 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f2067-5978-428c-b6a6-c8cfe4b4f7ad None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:16.259871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:16.308196 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f2067-5978-428c-b6a6-c8cfe4b4f7ad None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:16.341119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7803bbe-4afc-4ae3-b9ab-82a55087417a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:16.341738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7803bbe-4afc-4ae3-b9ab-82a55087417a None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:16.342037 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7803bbe-4afc-4ae3-b9ab-82a55087417a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:16.342309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7803bbe-4afc-4ae3-b9ab-82a55087417a None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:16.342986 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7803bbe-4afc-4ae3-b9ab-82a55087417a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:16.344416 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7803bbe-4afc-4ae3-b9ab-82a55087417a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:16.447955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 601/1205] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 10 16:58:16 2022] POST /identity/v3/users => generated 251 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:58:18.482211 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd059a-0296-4ea8-adb8-030dfe10ccf9 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:18.482781 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd059a-0296-4ea8-adb8-030dfe10ccf9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:18.482781 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd059a-0296-4ea8-adb8-030dfe10ccf9 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:18.484123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 605/1206] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:58:18.489477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24824a3d-5daf-418b-9fcb-ce0a2dbb7c86 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:18.496920 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24824a3d-5daf-418b-9fcb-ce0a2dbb7c86 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:18.496920 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24824a3d-5daf-418b-9fcb-ce0a2dbb7c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:18.508516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-24824a3d-5daf-418b-9fcb-ce0a2dbb7c86 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:58:18.509691 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-24824a3d-5daf-418b-9fcb-ce0a2dbb7c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:18.576355 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 602/1207] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:58:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:58:18.599378 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfd059a-0296-4ea8-adb8-030dfe10ccf9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:18.703599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfd059a-0296-4ea8-adb8-030dfe10ccf9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:18.737056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576ffa7-aee5-406e-a59e-55d53fe44407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:18.737594 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576ffa7-aee5-406e-a59e-55d53fe44407 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:18.737869 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576ffa7-aee5-406e-a59e-55d53fe44407 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:18.738140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576ffa7-aee5-406e-a59e-55d53fe44407 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:18.749156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8576ffa7-aee5-406e-a59e-55d53fe44407 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:58:18.753824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-8576ffa7-aee5-406e-a59e-55d53fe44407 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:58:18.753824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:18.753824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:18.753824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:18.753824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:18.753824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:18.753824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:18.753824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:18.753824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:18.753824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:18.753824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:18.753824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:18.753824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:58:18.753824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:58:18.753824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:58:18.753824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:58:18.755669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:58:18.755669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:58:18.755669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:58:18.755669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:58:18.755669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:18.755669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:18.755669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:18.755669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:18.755669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:18.755669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:18.755669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:18.755669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:18.755669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:18.755669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:18.755669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:18.755669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:18.756524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:18.756524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:18.756524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:18.756524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:18.756524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:58:18.756524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:58:18.756524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:58:18.756524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:58:18.756524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:58:18.756524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:58:18.756524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:58:18.756524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:58:18.756524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 606/1208] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 10 16:58:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:18.761738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24824a3d-5daf-418b-9fcb-ce0a2dbb7c86 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:18.801148 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:18.858464 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24824a3d-5daf-418b-9fcb-ce0a2dbb7c86 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:18.893109 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11217a-206d-4ad4-a599-ae7d363577ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:18.893712 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11217a-206d-4ad4-a599-ae7d363577ca None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:18.893984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11217a-206d-4ad4-a599-ae7d363577ca None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:18.894250 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11217a-206d-4ad4-a599-ae7d363577ca None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:18.894947 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf11217a-206d-4ad4-a599-ae7d363577ca None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:18.895246 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf11217a-206d-4ad4-a599-ae7d363577ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:18.896604 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf11217a-206d-4ad4-a599-ae7d363577ca None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:18.905115 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 603/1209] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 10 16:58:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:58:18.910696 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576ffa7-aee5-406e-a59e-55d53fe44407 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:18.961167 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:19.012644 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576ffa7-aee5-406e-a59e-55d53fe44407 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:19.045901 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935367db-58c3-4a2d-b95f-ea10e177405b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:19.046548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935367db-58c3-4a2d-b95f-ea10e177405b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:19.046827 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935367db-58c3-4a2d-b95f-ea10e177405b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:19.047093 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935367db-58c3-4a2d-b95f-ea10e177405b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:19.059244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935367db-58c3-4a2d-b95f-ea10e177405b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:19.063064 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935367db-58c3-4a2d-b95f-ea10e177405b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:19.075943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-935367db-58c3-4a2d-b95f-ea10e177405b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:58:19.075943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:19.075943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:19.075943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:19.075943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:19.075943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:19.075943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:19.075943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:19.075943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:19.075943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:19.075943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:19.075943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:19.075943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:58:19.075943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:58:19.075943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:58:19.075943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:58:19.077363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:19.077363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:19.077363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:19.077363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:19.077363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:19.077363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:19.077363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:19.077363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:19.077363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:19.077363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:19.077363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:19.077363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:19.077363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:19.077363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:19.077363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:19.077363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:19.077363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:58:19.078647 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:58:19.078647 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:58:19.078647 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:58:19.078647 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 607/1210] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:58:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:19.083036 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11217a-206d-4ad4-a599-ae7d363577ca None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:19.179464 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:19.182792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11217a-206d-4ad4-a599-ae7d363577ca None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:19.218830 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060d50f-ae22-4156-b00b-0d785289f9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:19.219428 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060d50f-ae22-4156-b00b-0d785289f9b1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:19.219704 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060d50f-ae22-4156-b00b-0d785289f9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:19.219972 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060d50f-ae22-4156-b00b-0d785289f9b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:19.220613 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060d50f-ae22-4156-b00b-0d785289f9b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:19.220912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060d50f-ae22-4156-b00b-0d785289f9b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:19.222267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060d50f-ae22-4156-b00b-0d785289f9b1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:19.234130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 604/1211] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:58:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:19.240029 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935367db-58c3-4a2d-b95f-ea10e177405b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:19.339110 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935367db-58c3-4a2d-b95f-ea10e177405b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:19.340695 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:19.372793 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7765e7-1deb-4f0c-8831-3e55c91b40e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:19.374166 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7765e7-1deb-4f0c-8831-3e55c91b40e5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:19.374166 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7765e7-1deb-4f0c-8831-3e55c91b40e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:19.374166 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7765e7-1deb-4f0c-8831-3e55c91b40e5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:19.381310 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7765e7-1deb-4f0c-8831-3e55c91b40e5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:19.382764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7765e7-1deb-4f0c-8831-3e55c91b40e5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:19.393656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-7b7765e7-1deb-4f0c-8831-3e55c91b40e5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 16:58:19.393656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:19.393656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:19.393656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:19.393656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:19.393656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:19.393656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:19.393656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:19.393656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:19.393656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:19.393656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:19.393656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:19.393656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:58:19.393656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:58:19.393656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:58:19.393656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:58:19.394843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:19.394843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:19.394843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:58:19.394843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:58:19.394843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:58:19.394843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:58:19.394843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:19.394843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:19.394843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:19.394843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:19.394843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:19.394843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:19.394843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:19.394843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:19.394843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:19.394843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:19.394843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:19.394843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:19.395996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:19.395996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:19.395996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:58:19.395996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:58:19.395996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:58:19.395996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:58:19.395996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:58:19.395996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:58:19.395996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 16:58:19.395996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:58:19.395996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 608/1212] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:58:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:19.401007 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060d50f-ae22-4156-b00b-0d785289f9b1 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:19.498816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060d50f-ae22-4156-b00b-0d785289f9b1 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:19.532435 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d446a03-212b-4343-baef-0b8a96064579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:19.533030 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d446a03-212b-4343-baef-0b8a96064579 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:19.533301 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d446a03-212b-4343-baef-0b8a96064579 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:19.533566 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d446a03-212b-4343-baef-0b8a96064579 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:19.534279 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d446a03-212b-4343-baef-0b8a96064579 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:19.534609 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d446a03-212b-4343-baef-0b8a96064579 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:19.536020 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d446a03-212b-4343-baef-0b8a96064579 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:19.553964 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 605/1213] 10.222.0.45 () {56 vars in 1143 bytes} [Wed Aug 10 16:58:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:19.560124 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7765e7-1deb-4f0c-8831-3e55c91b40e5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:19.662930 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7765e7-1deb-4f0c-8831-3e55c91b40e5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:19.696947 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa61f12-3a27-418c-b49e-bcafe19f87cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:19.697547 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa61f12-3a27-418c-b49e-bcafe19f87cc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:19.697822 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa61f12-3a27-418c-b49e-bcafe19f87cc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:19.698087 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa61f12-3a27-418c-b49e-bcafe19f87cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:19.729117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:19.765258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa61f12-3a27-418c-b49e-bcafe19f87cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:19.766738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa61f12-3a27-418c-b49e-bcafe19f87cc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:19.867575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 609/1214] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:58:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:58:19.873926 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d446a03-212b-4343-baef-0b8a96064579 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:19.893292 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:19.974481 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d446a03-212b-4343-baef-0b8a96064579 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:20.009354 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790fd6ba-ecfe-484a-9876-1490b0ea3f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:20.009976 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790fd6ba-ecfe-484a-9876-1490b0ea3f3c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:20.010248 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790fd6ba-ecfe-484a-9876-1490b0ea3f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:20.010597 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790fd6ba-ecfe-484a-9876-1490b0ea3f3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:20.020309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790fd6ba-ecfe-484a-9876-1490b0ea3f3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:20.021725 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790fd6ba-ecfe-484a-9876-1490b0ea3f3c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:20.036541 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-790fd6ba-ecfe-484a-9876-1490b0ea3f3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:58:20.036541 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:20.036541 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:20.036541 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:20.036541 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:20.036541 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:20.036541 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:20.036541 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:20.036541 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:20.036541 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:20.036541 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:20.036541 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:20.036541 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:58:20.036541 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:58:20.036541 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:58:20.036541 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:58:20.037722 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:20.037722 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:20.037722 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:20.037722 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:20.037722 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:20.037722 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:20.037722 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:20.037722 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:20.037722 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:20.037722 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:20.037722 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:20.037722 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:20.037722 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:20.037722 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:20.037722 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:20.037722 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:20.037722 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:58:20.038866 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:58:20.038866 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:58:20.038866 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:58:20.038866 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 606/1215] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:58:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:20.043570 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa61f12-3a27-418c-b49e-bcafe19f87cc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:20.104343 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:20.142275 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa61f12-3a27-418c-b49e-bcafe19f87cc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:20.175613 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457aa606-5243-41a9-a038-5dc213645f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:20.176211 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457aa606-5243-41a9-a038-5dc213645f5d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:20.176525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457aa606-5243-41a9-a038-5dc213645f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:20.176794 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457aa606-5243-41a9-a038-5dc213645f5d None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:20.177461 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457aa606-5243-41a9-a038-5dc213645f5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:20.177761 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457aa606-5243-41a9-a038-5dc213645f5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:20.179159 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457aa606-5243-41a9-a038-5dc213645f5d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:20.191756 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 610/1216] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:58:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:20.198017 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790fd6ba-ecfe-484a-9876-1490b0ea3f3c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:20.266121 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:20.296599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790fd6ba-ecfe-484a-9876-1490b0ea3f3c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:20.330267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb7106-f63b-4599-91b2-441a07c6af82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:20.330916 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb7106-f63b-4599-91b2-441a07c6af82 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:20.331260 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb7106-f63b-4599-91b2-441a07c6af82 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:20.331505 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb7106-f63b-4599-91b2-441a07c6af82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:20.338227 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eb7106-f63b-4599-91b2-441a07c6af82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:20.339689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eb7106-f63b-4599-91b2-441a07c6af82 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:20.351002 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-a9eb7106-f63b-4599-91b2-441a07c6af82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:58:20.351002 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:20.351002 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:20.351002 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:20.351002 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:20.351002 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:20.351002 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:20.351002 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:20.351002 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:20.351002 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:20.351002 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:20.351002 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:20.351002 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:58:20.351002 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:58:20.351002 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:58:20.352195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:58:20.352195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:20.352195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:20.352195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:20.352195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:20.352195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:20.352195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:20.352195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:20.352195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:20.352195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:20.352195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:20.352195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:20.352195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:20.352195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:20.352195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:20.352195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:20.352195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:20.353558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:58:20.353558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:58:20.353558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:58:20.353558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:58:20.353558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:58:20.353558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:58:20.353558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:58:20.353558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:58:20.353558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 607/1217] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:58:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:20.358260 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457aa606-5243-41a9-a038-5dc213645f5d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:20.458891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457aa606-5243-41a9-a038-5dc213645f5d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:20.480768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:20.492019 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd6fbe-0d47-4177-8611-72e4b547df55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:20.492617 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd6fbe-0d47-4177-8611-72e4b547df55 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:20.493554 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd6fbe-0d47-4177-8611-72e4b547df55 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:20.493878 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd6fbe-0d47-4177-8611-72e4b547df55 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:20.494597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cd6fbe-0d47-4177-8611-72e4b547df55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:20.494899 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cd6fbe-0d47-4177-8611-72e4b547df55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:20.496321 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cd6fbe-0d47-4177-8611-72e4b547df55 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:20.508983 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 611/1218] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:58:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:20.515390 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb7106-f63b-4599-91b2-441a07c6af82 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:20.620412 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb7106-f63b-4599-91b2-441a07c6af82 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:20.648277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:20.653913 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3052e-d519-4254-9eac-6bc3741bb924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:20.654547 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3052e-d519-4254-9eac-6bc3741bb924 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:20.654830 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3052e-d519-4254-9eac-6bc3741bb924 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:20.655098 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3052e-d519-4254-9eac-6bc3741bb924 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:20.655814 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3052e-d519-4254-9eac-6bc3741bb924 None admin] RBAC: Adding query filter params (role.id=303e0fba379a41019ecc2e119d87fb1b, scope.project.id=d11f33af53374dffa61132642a7770f4, user.id=8cd39d9a583a4d73a5654017be106019) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:20.656116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3052e-d519-4254-9eac-6bc3741bb924 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:20.657532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3052e-d519-4254-9eac-6bc3741bb924 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:20.662755 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 608/1219] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:58:20 2022] GET /identity/v3/role_assignments?user.id=8cd39d9a583a4d73a5654017be106019&scope.project.id=d11f33af53374dffa61132642a7770f4&role.id=303e0fba379a41019ecc2e119d87fb1b => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:22.660055 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edde3e-538a-4f47-a576-def455197749 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:22.660631 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edde3e-538a-4f47-a576-def455197749 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:22.660631 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edde3e-538a-4f47-a576-def455197749 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:22.661933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 612/1220] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:58:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:58:22.667345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846368e-5e6f-44f6-a8da-9caa83632d19 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:22.670586 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846368e-5e6f-44f6-a8da-9caa83632d19 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:22.670586 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846368e-5e6f-44f6-a8da-9caa83632d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:22.686625 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b846368e-5e6f-44f6-a8da-9caa83632d19 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:58:22.687801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b846368e-5e6f-44f6-a8da-9caa83632d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:22.755224 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 609/1221] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:58:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:58:22.778595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0edde3e-538a-4f47-a576-def455197749 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:22.878437 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0edde3e-538a-4f47-a576-def455197749 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:22.911645 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524e5d7-b017-4547-9417-fb8a7859815d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:22.912271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524e5d7-b017-4547-9417-fb8a7859815d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:22.912547 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524e5d7-b017-4547-9417-fb8a7859815d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:22.912817 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524e5d7-b017-4547-9417-fb8a7859815d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:22.924473 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524e5d7-b017-4547-9417-fb8a7859815d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:58:22.929131 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-9524e5d7-b017-4547-9417-fb8a7859815d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:58:22.929131 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:22.929131 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:22.929131 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:22.929131 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:22.929131 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:22.929131 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:22.929131 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:22.929131 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:22.929131 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:22.929131 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:22.929131 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:22.929131 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:58:22.929131 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:58:22.929131 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:58:22.929131 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:58:22.930325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:58:22.930325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:58:22.930325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:58:22.930325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:58:22.930325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:22.930325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:22.930325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:22.930325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:22.930325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:22.930325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:22.930325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:22.930325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:22.930325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:22.930325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:22.930325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:22.930325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:22.932287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:22.932287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:22.932287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:22.932287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:22.932287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:58:22.932287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:58:22.932287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:58:22.932287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:58:22.932287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:58:22.932287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:58:22.932287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:58:22.932287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:58:22.932287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 613/1222] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 10 16:58:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:22.936926 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846368e-5e6f-44f6-a8da-9caa83632d19 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:23.032753 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846368e-5e6f-44f6-a8da-9caa83632d19 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:23.057295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:23.067502 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132d681-f172-46ff-a745-a844569a9090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:23.068101 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132d681-f172-46ff-a745-a844569a9090 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:23.068381 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132d681-f172-46ff-a745-a844569a9090 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:23.068653 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132d681-f172-46ff-a745-a844569a9090 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:23.069310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c132d681-f172-46ff-a745-a844569a9090 None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:23.069606 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c132d681-f172-46ff-a745-a844569a9090 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:23.071001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c132d681-f172-46ff-a745-a844569a9090 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:23.080193 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 610/1223] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 10 16:58:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:23.086019 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524e5d7-b017-4547-9417-fb8a7859815d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:23.130153 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:23.185074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524e5d7-b017-4547-9417-fb8a7859815d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:23.218093 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d943c55-e8fa-4da2-94ba-08efe011ff20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:23.218738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d943c55-e8fa-4da2-94ba-08efe011ff20 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:23.219018 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d943c55-e8fa-4da2-94ba-08efe011ff20 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:23.219289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d943c55-e8fa-4da2-94ba-08efe011ff20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:23.229658 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d943c55-e8fa-4da2-94ba-08efe011ff20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:23.231127 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d943c55-e8fa-4da2-94ba-08efe011ff20 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:23.245614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-9d943c55-e8fa-4da2-94ba-08efe011ff20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:58:23.245614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:23.245614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:23.245614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:23.245614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:23.245614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:23.245614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:23.245614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:23.245614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:23.245614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:23.245614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:23.245614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:23.245614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:58:23.245614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:58:23.245614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:58:23.245614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:58:23.246862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:23.246862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:23.246862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:23.246862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:23.246862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:23.246862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:23.246862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:23.246862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:23.246862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:23.246862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:23.246862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:23.246862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:23.246862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:23.246862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:23.246862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:23.246862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:23.246862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:58:23.247943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:58:23.247943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:58:23.247943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:58:23.247943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 614/1224] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:58:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:23.252579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132d681-f172-46ff-a745-a844569a9090 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:23.352369 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132d681-f172-46ff-a745-a844569a9090 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:23.386258 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def1d44-cf12-4756-985e-c4da54e62b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:23.386799 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def1d44-cf12-4756-985e-c4da54e62b29 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:23.387067 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def1d44-cf12-4756-985e-c4da54e62b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:23.387330 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def1d44-cf12-4756-985e-c4da54e62b29 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:23.387963 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4def1d44-cf12-4756-985e-c4da54e62b29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:23.388252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4def1d44-cf12-4756-985e-c4da54e62b29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:23.389618 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4def1d44-cf12-4756-985e-c4da54e62b29 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:23.402305 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 611/1225] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:23.408295 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d943c55-e8fa-4da2-94ba-08efe011ff20 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:23.502509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:23.509619 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d943c55-e8fa-4da2-94ba-08efe011ff20 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:23.542914 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d53028-1a8e-46a7-b425-d05e8612e3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:23.543527 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d53028-1a8e-46a7-b425-d05e8612e3f4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:23.543804 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d53028-1a8e-46a7-b425-d05e8612e3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:23.544073 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d53028-1a8e-46a7-b425-d05e8612e3f4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:23.551196 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d53028-1a8e-46a7-b425-d05e8612e3f4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:23.552621 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d53028-1a8e-46a7-b425-d05e8612e3f4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:23.563258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-28d53028-1a8e-46a7-b425-d05e8612e3f4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 16:58:23.563258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:23.563258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:23.563258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:23.563258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:23.563258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:23.563258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:23.563258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:23.563258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:23.563258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:23.563258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:23.563258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:23.563258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:58:23.563258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:58:23.563258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:58:23.563258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:58:23.564285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:23.564285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:23.564285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:58:23.564285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:58:23.564285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:58:23.564285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:58:23.564285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:23.564285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:23.564285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:23.564285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:23.564285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:23.564285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:23.564285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:23.564285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:23.564285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:23.564285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:23.564285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:23.564285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:23.565346 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:23.565346 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:23.565346 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:58:23.565346 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:58:23.565346 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:58:23.565346 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:58:23.565346 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:58:23.565346 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:58:23.565346 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 16:58:23.565346 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:58:23.565346 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 615/1226] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:58:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:23.570624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def1d44-cf12-4756-985e-c4da54e62b29 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:23.617527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:23.671502 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def1d44-cf12-4756-985e-c4da54e62b29 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:23.705280 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b693c-80d3-4acf-bb17-08647c38f020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:23.705870 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b693c-80d3-4acf-bb17-08647c38f020 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:23.706146 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b693c-80d3-4acf-bb17-08647c38f020 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:23.706471 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b693c-80d3-4acf-bb17-08647c38f020 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:23.707190 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b693c-80d3-4acf-bb17-08647c38f020 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:23.707491 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b693c-80d3-4acf-bb17-08647c38f020 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:23.708960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b693c-80d3-4acf-bb17-08647c38f020 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:23.726521 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 612/1227] 10.222.0.45 () {56 vars in 1143 bytes} [Wed Aug 10 16:58:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:58:23.732740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d53028-1a8e-46a7-b425-d05e8612e3f4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:23.831484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d53028-1a8e-46a7-b425-d05e8612e3f4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:23.864655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb047c0-59f0-4611-a1d8-d89ff5dea472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:23.866109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb047c0-59f0-4611-a1d8-d89ff5dea472 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:23.866109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb047c0-59f0-4611-a1d8-d89ff5dea472 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:23.866109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb047c0-59f0-4611-a1d8-d89ff5dea472 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:23.877477 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb047c0-59f0-4611-a1d8-d89ff5dea472 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:23.878952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb047c0-59f0-4611-a1d8-d89ff5dea472 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:23.892827 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4fb047c0-59f0-4611-a1d8-d89ff5dea472 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:58:23.892827 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:23.892827 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:23.892827 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:23.892827 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:23.892827 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:23.892827 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:23.892827 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:23.892827 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:23.892827 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:23.892827 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:23.892827 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:23.892827 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:58:23.892827 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:58:23.892827 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:58:23.892827 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:58:23.895632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:23.895632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:23.895632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:23.895632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:23.895632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:23.895632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:23.895632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:23.895632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:23.895632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:23.895632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:23.895632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:23.895632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:23.895632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:23.895632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:23.895632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:23.895632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:23.895632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:58:23.896586 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:58:23.896586 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:58:23.896586 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:58:23.896586 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 616/1228] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:58:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:23.900433 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b693c-80d3-4acf-bb17-08647c38f020 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:23.999501 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b693c-80d3-4acf-bb17-08647c38f020 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:24.001729 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:24.032946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49adc67a-ed14-45a8-8654-998000fee41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:24.033535 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49adc67a-ed14-45a8-8654-998000fee41f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:24.036300 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49adc67a-ed14-45a8-8654-998000fee41f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:24.036300 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49adc67a-ed14-45a8-8654-998000fee41f None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:24.036300 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49adc67a-ed14-45a8-8654-998000fee41f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:24.036300 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49adc67a-ed14-45a8-8654-998000fee41f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:24.036300 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49adc67a-ed14-45a8-8654-998000fee41f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:24.047955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 613/1229] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:24.053903 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb047c0-59f0-4611-a1d8-d89ff5dea472 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:24.063208 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:24.154724 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb047c0-59f0-4611-a1d8-d89ff5dea472 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:24.187103 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958636a-3c86-484c-96c9-dd1945fa298f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:24.187789 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958636a-3c86-484c-96c9-dd1945fa298f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:24.188068 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958636a-3c86-484c-96c9-dd1945fa298f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:24.188336 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958636a-3c86-484c-96c9-dd1945fa298f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:24.194976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5958636a-3c86-484c-96c9-dd1945fa298f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:24.197242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5958636a-3c86-484c-96c9-dd1945fa298f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:24.208323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5958636a-3c86-484c-96c9-dd1945fa298f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:58:24.208323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:24.208323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:24.208323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:24.208323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:24.208323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:24.208323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:24.208323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:24.208323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:24.208323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:24.208323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:24.208323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:24.208323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:58:24.208323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:58:24.208323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:58:24.211250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:58:24.211250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:24.211250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:24.211250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:24.211250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:24.211250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:24.211250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:24.211250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:24.211250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:24.211250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:24.211250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:24.211250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:24.211250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:24.211250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:24.211250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:24.211250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:24.211250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:24.213125 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:58:24.213125 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:58:24.213125 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:58:24.213125 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:58:24.213125 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:58:24.213125 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:58:24.213125 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:58:24.213125 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:58:24.213125 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 617/1230] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:58:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:24.216251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49adc67a-ed14-45a8-8654-998000fee41f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:24.313623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49adc67a-ed14-45a8-8654-998000fee41f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:24.348806 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6821e-4040-47d2-b3ae-e4c111cd7cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:24.349693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6821e-4040-47d2-b3ae-e4c111cd7cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:24.349693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6821e-4040-47d2-b3ae-e4c111cd7cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:24.350002 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6821e-4040-47d2-b3ae-e4c111cd7cc0 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:24.350730 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6821e-4040-47d2-b3ae-e4c111cd7cc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:24.351033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6821e-4040-47d2-b3ae-e4c111cd7cc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:24.352609 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6821e-4040-47d2-b3ae-e4c111cd7cc0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:24.364678 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 614/1231] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:58:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:24.370974 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958636a-3c86-484c-96c9-dd1945fa298f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:24.437543 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:24.470726 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958636a-3c86-484c-96c9-dd1945fa298f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:24.502107 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b28a321-e9b9-4905-b6fa-d8e17e8dcab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:24.502891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28a321-e9b9-4905-b6fa-d8e17e8dcab0 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:24.503158 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28a321-e9b9-4905-b6fa-d8e17e8dcab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:24.503437 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28a321-e9b9-4905-b6fa-d8e17e8dcab0 None admin] PATH_INFO: `/v3/projects/d11f33af53374dffa61132642a7770f4/users/8cd39d9a583a4d73a5654017be106019/roles/303e0fba379a41019ecc2e119d87fb1b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:24.518494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b28a321-e9b9-4905-b6fa-d8e17e8dcab0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d11f33af53374dffa61132642a7770f4, user_id=8cd39d9a583a4d73a5654017be106019, role_id=303e0fba379a41019ecc2e119d87fb1b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:24.519969 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b28a321-e9b9-4905-b6fa-d8e17e8dcab0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:24.710012 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 618/1232] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 10 16:58:24 2022] PUT /identity/v3/projects/d11f33af53374dffa61132642a7770f4/users/8cd39d9a583a4d73a5654017be106019/roles/303e0fba379a41019ecc2e119d87fb1b => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:58:26.709232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8340c-9161-4056-9264-34996aac0aa5 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:26.709781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8340c-9161-4056-9264-34996aac0aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:26.709781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8340c-9161-4056-9264-34996aac0aa5 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:26.711166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 615/1233] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:58:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:58:26.716468 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51821119-be3d-4627-a0b7-569c6ece1955 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:26.716966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51821119-be3d-4627-a0b7-569c6ece1955 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:26.717063 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51821119-be3d-4627-a0b7-569c6ece1955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:26.735030 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-51821119-be3d-4627-a0b7-569c6ece1955 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:58:26.736202 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-51821119-be3d-4627-a0b7-569c6ece1955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:26.806283 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 619/1234] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:58:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:58:26.830071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8340c-9161-4056-9264-34996aac0aa5 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:26.855228 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:26.928571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8340c-9161-4056-9264-34996aac0aa5 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:26.962822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba38da-4de1-4b19-9c28-2f10e073d89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:26.963476 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba38da-4de1-4b19-9c28-2f10e073d89f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:26.963753 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba38da-4de1-4b19-9c28-2f10e073d89f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:26.964021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba38da-4de1-4b19-9c28-2f10e073d89f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:26.973861 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba38da-4de1-4b19-9c28-2f10e073d89f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:58:26.979107 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-73ba38da-4de1-4b19-9c28-2f10e073d89f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:58:26.979107 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:26.979107 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:26.979107 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:26.979107 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:26.979107 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:26.979107 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:26.979107 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:26.979107 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:26.979107 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:26.979107 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:26.979107 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:26.979107 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:58:26.979107 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:58:26.979107 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:58:26.979107 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:58:26.980218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:58:26.980218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:58:26.980218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:58:26.980218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:58:26.980218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:26.980218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:26.980218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:26.980218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:26.980218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:26.980218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:26.980218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:26.980218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:26.980218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:26.980218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:26.980218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:26.980218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:26.981159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:26.981159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:26.981159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:26.981159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:26.981159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:58:26.981159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:58:26.981159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:58:26.981159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:58:26.981159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:58:26.981159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:58:26.981159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 16:58:26.981159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:58:26.981159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 616/1235] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 10 16:58:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:26.986472 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51821119-be3d-4627-a0b7-569c6ece1955 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:27.031888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:27.082371 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51821119-be3d-4627-a0b7-569c6ece1955 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:27.116889 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db20a0bb-61a0-44cb-8615-174bad91a9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:27.117487 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20a0bb-61a0-44cb-8615-174bad91a9d2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:27.117770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20a0bb-61a0-44cb-8615-174bad91a9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:27.118070 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20a0bb-61a0-44cb-8615-174bad91a9d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:27.118759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db20a0bb-61a0-44cb-8615-174bad91a9d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:27.119058 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db20a0bb-61a0-44cb-8615-174bad91a9d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:27.120417 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db20a0bb-61a0-44cb-8615-174bad91a9d2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:27.127991 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 620/1236] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 10 16:58:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:27.133799 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba38da-4de1-4b19-9c28-2f10e073d89f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:27.221812 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:27.237286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba38da-4de1-4b19-9c28-2f10e073d89f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:27.269671 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca241d-dd77-4529-bd76-56e09c49eec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:27.270284 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca241d-dd77-4529-bd76-56e09c49eec8 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:27.270605 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca241d-dd77-4529-bd76-56e09c49eec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:27.270873 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca241d-dd77-4529-bd76-56e09c49eec8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:27.281809 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca241d-dd77-4529-bd76-56e09c49eec8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:27.283265 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca241d-dd77-4529-bd76-56e09c49eec8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:27.297290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-1aca241d-dd77-4529-bd76-56e09c49eec8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:58:27.297290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:27.297290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:27.297290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:27.297290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:27.297290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:27.297290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:27.297290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:27.297290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:27.297290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:27.297290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:27.297290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:27.297290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:58:27.297290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:58:27.297290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:58:27.297290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:58:27.298456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:27.298456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:27.298456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:27.298456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:27.298456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:27.298456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:27.298456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:27.298456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:27.298456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:27.298456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:27.298456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:27.298456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:27.298456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:27.298456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:27.298456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:27.298456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:27.298456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:58:27.299605 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:58:27.299605 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:58:27.299605 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:58:27.299605 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 617/1237] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:58:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:27.304227 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db20a0bb-61a0-44cb-8615-174bad91a9d2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:27.403699 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db20a0bb-61a0-44cb-8615-174bad91a9d2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:27.412425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:27.439493 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e0053-916b-4a8b-94f1-96de75887ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:27.440094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e0053-916b-4a8b-94f1-96de75887ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:27.440367 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e0053-916b-4a8b-94f1-96de75887ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:27.440635 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e0053-916b-4a8b-94f1-96de75887ee7 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:27.441266 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e0053-916b-4a8b-94f1-96de75887ee7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:27.441599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e0053-916b-4a8b-94f1-96de75887ee7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:27.443037 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e0053-916b-4a8b-94f1-96de75887ee7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:27.454803 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 621/1238] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:58:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:27.460653 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca241d-dd77-4529-bd76-56e09c49eec8 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:27.573094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca241d-dd77-4529-bd76-56e09c49eec8 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:27.607496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04066be-26bd-451c-91ef-f3613b1976b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:27.608415 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04066be-26bd-451c-91ef-f3613b1976b5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:27.608415 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04066be-26bd-451c-91ef-f3613b1976b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:27.608624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04066be-26bd-451c-91ef-f3613b1976b5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:27.611377 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:27.616668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04066be-26bd-451c-91ef-f3613b1976b5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:27.618688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04066be-26bd-451c-91ef-f3613b1976b5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:27.630287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-b04066be-26bd-451c-91ef-f3613b1976b5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 16:58:27.630287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:27.630287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:27.630287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:27.630287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:27.630287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:27.630287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:27.630287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:27.630287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:27.630287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:27.630287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:27.630287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:27.630287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:58:27.630287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:58:27.630287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:58:27.630287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:58:27.631461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:27.631461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:27.631461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:58:27.631461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:58:27.631461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:58:27.631461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:58:27.631461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:27.631461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:27.631461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:27.631461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:27.631461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:27.631461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:27.631461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:27.631461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:27.631461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:27.631461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:27.631461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:27.631461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:27.633957 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:27.633957 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:27.633957 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:58:27.633957 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:58:27.633957 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:58:27.633957 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:58:27.633957 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:58:27.633957 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:58:27.633957 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 16:58:27.633957 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:58:27.633957 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 618/1239] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:58:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:27.637478 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e0053-916b-4a8b-94f1-96de75887ee7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:27.734770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e0053-916b-4a8b-94f1-96de75887ee7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:27.768899 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727db0fd-4950-4537-9f47-cb612eefe2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:27.769490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727db0fd-4950-4537-9f47-cb612eefe2b5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:27.769761 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727db0fd-4950-4537-9f47-cb612eefe2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:27.770025 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727db0fd-4950-4537-9f47-cb612eefe2b5 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:27.770779 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727db0fd-4950-4537-9f47-cb612eefe2b5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:27.775765 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727db0fd-4950-4537-9f47-cb612eefe2b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:27.775765 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727db0fd-4950-4537-9f47-cb612eefe2b5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:27.789512 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 622/1240] 10.222.0.45 () {56 vars in 1143 bytes} [Wed Aug 10 16:58:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:27.795969 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04066be-26bd-451c-91ef-f3613b1976b5 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:27.895103 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04066be-26bd-451c-91ef-f3613b1976b5 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:27.930581 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c9180e-744a-4e78-91ed-6028a9b416b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:27.932048 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9180e-744a-4e78-91ed-6028a9b416b9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:27.932048 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9180e-744a-4e78-91ed-6028a9b416b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:27.932243 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9180e-744a-4e78-91ed-6028a9b416b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:27.993173 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:27.999601 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c9180e-744a-4e78-91ed-6028a9b416b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:28.001061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c9180e-744a-4e78-91ed-6028a9b416b9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:28.098038 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 619/1241] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:58:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:58:28.104363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727db0fd-4950-4537-9f47-cb612eefe2b5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:28.133243 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:28.205391 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727db0fd-4950-4537-9f47-cb612eefe2b5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:28.237701 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43b51f-dab7-4d2e-82cf-c765a1360ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:28.238315 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43b51f-dab7-4d2e-82cf-c765a1360ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:28.238617 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43b51f-dab7-4d2e-82cf-c765a1360ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:28.238880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43b51f-dab7-4d2e-82cf-c765a1360ab6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:28.248585 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d43b51f-dab7-4d2e-82cf-c765a1360ab6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:28.250017 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d43b51f-dab7-4d2e-82cf-c765a1360ab6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:28.264774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-8d43b51f-dab7-4d2e-82cf-c765a1360ab6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:58:28.264774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:28.264774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:28.264774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:28.264774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:28.264774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:28.264774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:28.264774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:28.264774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:28.264774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:28.264774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:28.264774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:28.264774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:58:28.264774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:58:28.264774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:58:28.264774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:58:28.266072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:28.266072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:28.266072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:28.266072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:28.266072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:28.266072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:28.266072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:28.266072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:28.266072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:28.266072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:28.266072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:28.266072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:28.266072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:28.266072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:28.266072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:28.266072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:28.266072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:58:28.267189 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:58:28.267189 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:58:28.267189 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:58:28.267189 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 623/1242] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:58:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:28.271799 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c9180e-744a-4e78-91ed-6028a9b416b9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:28.366551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:28.372854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c9180e-744a-4e78-91ed-6028a9b416b9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:28.405611 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173c169-51bd-4397-ab37-1f14ead8f9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:28.406206 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173c169-51bd-4397-ab37-1f14ead8f9e5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:28.406504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173c169-51bd-4397-ab37-1f14ead8f9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:28.406768 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173c169-51bd-4397-ab37-1f14ead8f9e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:28.408076 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173c169-51bd-4397-ab37-1f14ead8f9e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:28.408076 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173c169-51bd-4397-ab37-1f14ead8f9e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:28.409037 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173c169-51bd-4397-ab37-1f14ead8f9e5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:28.420721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 620/1243] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:58:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:28.426506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43b51f-dab7-4d2e-82cf-c765a1360ab6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:28.504671 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:28.526113 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43b51f-dab7-4d2e-82cf-c765a1360ab6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:28.558624 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c1671-2fac-4273-bd9e-690ff353a851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:28.559253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c1671-2fac-4273-bd9e-690ff353a851 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:28.559528 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c1671-2fac-4273-bd9e-690ff353a851 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:28.559795 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c1671-2fac-4273-bd9e-690ff353a851 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:28.566310 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2c1671-2fac-4273-bd9e-690ff353a851 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:28.567769 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2c1671-2fac-4273-bd9e-690ff353a851 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:28.578590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2a2c1671-2fac-4273-bd9e-690ff353a851 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:58:28.578590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:28.578590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:28.578590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:28.578590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:28.578590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:28.578590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:28.578590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:28.578590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:28.578590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:28.578590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:28.578590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:28.578590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:58:28.578590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:58:28.578590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:58:28.579619 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:58:28.579619 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:28.579619 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:28.579619 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:28.579619 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:28.579619 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:28.579619 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:28.579619 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:28.579619 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:28.579619 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:28.579619 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:28.579619 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:28.579619 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:28.579619 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:28.579619 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:28.579619 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:28.579619 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:28.580609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:58:28.580609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:58:28.580609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:58:28.580609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:58:28.580609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:58:28.580609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:58:28.580609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:58:28.580609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:58:28.580609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 624/1244] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:58:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:28.585650 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173c169-51bd-4397-ab37-1f14ead8f9e5 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:28.685094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173c169-51bd-4397-ab37-1f14ead8f9e5 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:28.718053 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aebdc1f-2fda-4598-84f0-73fdd92e3083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:28.718695 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebdc1f-2fda-4598-84f0-73fdd92e3083 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:28.718953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebdc1f-2fda-4598-84f0-73fdd92e3083 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:28.719222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebdc1f-2fda-4598-84f0-73fdd92e3083 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:28.719901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aebdc1f-2fda-4598-84f0-73fdd92e3083 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:28.720203 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aebdc1f-2fda-4598-84f0-73fdd92e3083 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:28.721614 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aebdc1f-2fda-4598-84f0-73fdd92e3083 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:28.734119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 621/1245] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:58:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:28.740391 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c1671-2fac-4273-bd9e-690ff353a851 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:28.841838 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c1671-2fac-4273-bd9e-690ff353a851 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:28.875384 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd7933-6d9b-4d3d-92ae-98ef7e9e377f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:28.875976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd7933-6d9b-4d3d-92ae-98ef7e9e377f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:28.876253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd7933-6d9b-4d3d-92ae-98ef7e9e377f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:28.876523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd7933-6d9b-4d3d-92ae-98ef7e9e377f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:28.877234 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd7933-6d9b-4d3d-92ae-98ef7e9e377f None admin] RBAC: Adding query filter params (role.id=303e0fba379a41019ecc2e119d87fb1b, scope.project.id=d11f33af53374dffa61132642a7770f4, user.id=8cd39d9a583a4d73a5654017be106019) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:28.877577 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd7933-6d9b-4d3d-92ae-98ef7e9e377f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:28.879035 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd7933-6d9b-4d3d-92ae-98ef7e9e377f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:28.884836 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 625/1246] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:58:28 2022] GET /identity/v3/role_assignments?user.id=8cd39d9a583a4d73a5654017be106019&scope.project.id=d11f33af53374dffa61132642a7770f4&role.id=303e0fba379a41019ecc2e119d87fb1b => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:30.941403 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba60ec3-c572-4a1e-9273-e040a9ba0954 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:30.941954 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba60ec3-c572-4a1e-9273-e040a9ba0954 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:30.941954 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba60ec3-c572-4a1e-9273-e040a9ba0954 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:30.943322 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 622/1247] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:58:30.948767 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ec4ae-29b7-47ec-a136-d0d21f184a5a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:30.949194 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ec4ae-29b7-47ec-a136-d0d21f184a5a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:30.949307 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ec4ae-29b7-47ec-a136-d0d21f184a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:30.955556 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:30.968517 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5f8ec4ae-29b7-47ec-a136-d0d21f184a5a None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:58:30.969782 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5f8ec4ae-29b7-47ec-a136-d0d21f184a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:31.036355 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 626/1248] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:58:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:58:31.060522 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba60ec3-c572-4a1e-9273-e040a9ba0954 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:31.079150 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:31.161905 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba60ec3-c572-4a1e-9273-e040a9ba0954 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:31.195749 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbeff84-cae3-4eeb-85ce-e2f015c367bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:31.196802 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbeff84-cae3-4eeb-85ce-e2f015c367bf None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:31.197072 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbeff84-cae3-4eeb-85ce-e2f015c367bf None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:31.197337 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbeff84-cae3-4eeb-85ce-e2f015c367bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:31.207400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbeff84-cae3-4eeb-85ce-e2f015c367bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:58:31.212759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-ecbeff84-cae3-4eeb-85ce-e2f015c367bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:58:31.212759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:31.212759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:31.212759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:31.212759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:31.212759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:31.212759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:31.212759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:31.212759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:31.212759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:31.212759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:31.212759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:31.212759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:58:31.212759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:58:31.212759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:58:31.212759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:58:31.213896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:58:31.213896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:58:31.213896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:58:31.213896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:58:31.213896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:31.213896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:31.213896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:31.213896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:31.213896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:31.213896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:31.213896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:31.213896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:31.213896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:31.213896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:31.213896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:31.213896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:31.215130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:31.215130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:31.215130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:31.215130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:31.215130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:58:31.215130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:58:31.215130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:58:31.215130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:58:31.215130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:58:31.215130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:58:31.215130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:58:31.215130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:58:31.215130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 623/1249] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:58:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:31.220014 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ec4ae-29b7-47ec-a136-d0d21f184a5a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:31.318656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ec4ae-29b7-47ec-a136-d0d21f184a5a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:31.348995 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:31.354232 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7921e-0a56-435b-8158-2e2d2dc75a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:31.354890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7921e-0a56-435b-8158-2e2d2dc75a2b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:31.355156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7921e-0a56-435b-8158-2e2d2dc75a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:31.355422 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7921e-0a56-435b-8158-2e2d2dc75a2b None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:31.356129 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff7921e-0a56-435b-8158-2e2d2dc75a2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:31.356447 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff7921e-0a56-435b-8158-2e2d2dc75a2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:31.357814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff7921e-0a56-435b-8158-2e2d2dc75a2b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:31.365743 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 627/1250] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:58:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:31.371775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbeff84-cae3-4eeb-85ce-e2f015c367bf None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:31.450928 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:31.473190 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbeff84-cae3-4eeb-85ce-e2f015c367bf None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:31.506216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda777e-815e-4864-92a8-df0a58b9dcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:31.515062 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda777e-815e-4864-92a8-df0a58b9dcac None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:31.515062 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda777e-815e-4864-92a8-df0a58b9dcac None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:31.515062 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda777e-815e-4864-92a8-df0a58b9dcac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:31.518308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda777e-815e-4864-92a8-df0a58b9dcac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:31.519795 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda777e-815e-4864-92a8-df0a58b9dcac None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:31.533668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-5dda777e-815e-4864-92a8-df0a58b9dcac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:58:31.533668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:31.533668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:31.533668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:31.533668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:31.533668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:31.533668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:31.533668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:31.533668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:31.533668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:31.533668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:31.533668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:31.533668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:58:31.533668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:58:31.533668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:58:31.533668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:58:31.534914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:31.534914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:31.534914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:31.534914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:31.534914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:31.534914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:31.534914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:31.534914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:31.534914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:31.534914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:31.534914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:31.534914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:31.534914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:31.534914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:31.534914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:31.534914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:31.534914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:58:31.536008 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:58:31.536008 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:58:31.536008 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:58:31.536008 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 624/1251] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:58:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:31.540869 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7921e-0a56-435b-8158-2e2d2dc75a2b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:31.642803 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7921e-0a56-435b-8158-2e2d2dc75a2b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:31.675215 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e508b-d32b-4fda-8943-543ca5167dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:31.675808 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e508b-d32b-4fda-8943-543ca5167dea None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:31.676080 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e508b-d32b-4fda-8943-543ca5167dea None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:31.676347 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e508b-d32b-4fda-8943-543ca5167dea None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:31.676966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043e508b-d32b-4fda-8943-543ca5167dea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:31.677255 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043e508b-d32b-4fda-8943-543ca5167dea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:31.678639 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043e508b-d32b-4fda-8943-543ca5167dea None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:31.690395 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 628/1252] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:31.696721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda777e-815e-4864-92a8-df0a58b9dcac None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:31.795103 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda777e-815e-4864-92a8-df0a58b9dcac None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:31.820199 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:31.828708 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac95ec-a9b6-4fef-b8d7-9232f329a43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:31.830942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac95ec-a9b6-4fef-b8d7-9232f329a43b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:31.830942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac95ec-a9b6-4fef-b8d7-9232f329a43b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:31.830942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac95ec-a9b6-4fef-b8d7-9232f329a43b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:31.837343 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caac95ec-a9b6-4fef-b8d7-9232f329a43b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:31.838929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caac95ec-a9b6-4fef-b8d7-9232f329a43b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:31.849787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-caac95ec-a9b6-4fef-b8d7-9232f329a43b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 16:58:31.849787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:31.849787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:31.849787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:31.849787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:31.849787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:31.849787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:31.849787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:31.849787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:31.849787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:31.849787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:31.849787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:31.849787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:58:31.849787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:58:31.849787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:58:31.849787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:58:31.850949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:31.850949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:31.850949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:58:31.850949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:58:31.850949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:58:31.850949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:58:31.850949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:31.850949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:31.850949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:31.850949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:31.850949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:31.850949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:31.850949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:31.850949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:31.850949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:31.850949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:31.850949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:31.850949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:31.852094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:31.852094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:31.852094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:58:31.852094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:58:31.852094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:58:31.852094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:58:31.852094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:58:31.852094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:58:31.852094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 16:58:31.852094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:58:31.852094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 625/1253] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:58:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:31.857561 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e508b-d32b-4fda-8943-543ca5167dea None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:31.911764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:31.955626 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e508b-d32b-4fda-8943-543ca5167dea None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:31.990723 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a6805-5e20-4108-b0bc-1c5ece38546e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:31.991327 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a6805-5e20-4108-b0bc-1c5ece38546e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:31.991610 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a6805-5e20-4108-b0bc-1c5ece38546e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:31.991875 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a6805-5e20-4108-b0bc-1c5ece38546e None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:31.992589 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a6805-5e20-4108-b0bc-1c5ece38546e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:31.992965 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a6805-5e20-4108-b0bc-1c5ece38546e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:31.994961 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a6805-5e20-4108-b0bc-1c5ece38546e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:32.011787 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 629/1254] 10.222.0.45 () {56 vars in 1143 bytes} [Wed Aug 10 16:58:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:32.017607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac95ec-a9b6-4fef-b8d7-9232f329a43b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:32.120671 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac95ec-a9b6-4fef-b8d7-9232f329a43b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:32.153249 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d971bf2-9cf0-46c7-900d-c9f6e562d601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:32.153841 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d971bf2-9cf0-46c7-900d-c9f6e562d601 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:32.154116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d971bf2-9cf0-46c7-900d-c9f6e562d601 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:32.154406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d971bf2-9cf0-46c7-900d-c9f6e562d601 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:32.208182 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:32.220746 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d971bf2-9cf0-46c7-900d-c9f6e562d601 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:32.222183 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d971bf2-9cf0-46c7-900d-c9f6e562d601 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:32.317583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 626/1255] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:58:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:58:32.323783 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a6805-5e20-4108-b0bc-1c5ece38546e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:32.424126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a6805-5e20-4108-b0bc-1c5ece38546e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:32.437758 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:32.457872 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d219f940-2b4a-43c0-9cb2-927f61fc385f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:32.458505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219f940-2b4a-43c0-9cb2-927f61fc385f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:32.458780 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219f940-2b4a-43c0-9cb2-927f61fc385f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:32.459043 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219f940-2b4a-43c0-9cb2-927f61fc385f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:32.469223 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d219f940-2b4a-43c0-9cb2-927f61fc385f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:32.470688 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d219f940-2b4a-43c0-9cb2-927f61fc385f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:32.485575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d219f940-2b4a-43c0-9cb2-927f61fc385f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:58:32.485575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:32.485575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:32.485575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:32.485575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:32.485575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:32.485575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:32.485575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:32.485575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:32.485575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:32.485575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:32.485575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:32.485575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:58:32.485575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:58:32.485575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:58:32.485575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:58:32.486751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:32.486751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:32.486751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:32.486751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:32.486751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:32.486751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:32.486751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:32.486751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:32.486751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:32.486751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:32.486751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:32.486751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:32.486751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:32.486751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:32.486751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:32.486751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:32.486751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:58:32.487989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:58:32.487989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:58:32.487989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:58:32.487989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 630/1256] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:58:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:32.492459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d971bf2-9cf0-46c7-900d-c9f6e562d601 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:32.578248 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:32.590280 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d971bf2-9cf0-46c7-900d-c9f6e562d601 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:32.623442 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a67cc-449f-44ee-806a-c04934072d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:32.629134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a67cc-449f-44ee-806a-c04934072d17 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:32.629134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a67cc-449f-44ee-806a-c04934072d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:32.629134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a67cc-449f-44ee-806a-c04934072d17 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:32.629134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a67cc-449f-44ee-806a-c04934072d17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:32.629134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a67cc-449f-44ee-806a-c04934072d17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:32.629134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a67cc-449f-44ee-806a-c04934072d17 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:32.638747 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 627/1257] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:58:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:32.644454 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d219f940-2b4a-43c0-9cb2-927f61fc385f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:32.742326 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d219f940-2b4a-43c0-9cb2-927f61fc385f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:32.776816 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1814db-a8d8-4d28-82fb-bb642771bdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:32.777424 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1814db-a8d8-4d28-82fb-bb642771bdb8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:32.777698 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1814db-a8d8-4d28-82fb-bb642771bdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:32.777968 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1814db-a8d8-4d28-82fb-bb642771bdb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:32.784426 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1814db-a8d8-4d28-82fb-bb642771bdb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:32.785861 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1814db-a8d8-4d28-82fb-bb642771bdb8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:32.797117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0f1814db-a8d8-4d28-82fb-bb642771bdb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:58:32.797117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:32.797117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:32.797117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:32.797117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:32.797117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:32.797117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:32.797117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:32.797117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:32.797117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:32.797117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:32.797117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:32.797117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:58:32.797117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:58:32.797117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:58:32.801755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:58:32.801755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:32.801755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:32.801755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:32.801755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:32.801755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:32.801755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:32.801755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:32.801755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:32.801755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:32.801755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:32.801755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:32.801755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:32.801755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:32.801755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:32.801755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:32.801755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:32.802913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:58:32.802913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:58:32.802913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:58:32.802913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:58:32.802913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:58:32.802913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:58:32.802913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:58:32.802913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:58:32.802913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 631/1258] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:58:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:32.804296 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a67cc-449f-44ee-806a-c04934072d17 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:32.903747 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a67cc-449f-44ee-806a-c04934072d17 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:32.937536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f49004-8593-4c17-a6b5-f59d15678911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:32.938143 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f49004-8593-4c17-a6b5-f59d15678911 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:32.938448 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f49004-8593-4c17-a6b5-f59d15678911 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:32.938717 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f49004-8593-4c17-a6b5-f59d15678911 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:32.939407 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f49004-8593-4c17-a6b5-f59d15678911 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:32.939742 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f49004-8593-4c17-a6b5-f59d15678911 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:32.941159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f49004-8593-4c17-a6b5-f59d15678911 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:32.953948 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 628/1259] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:58:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:58:32.960175 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1814db-a8d8-4d28-82fb-bb642771bdb8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:32.985603 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:33.059657 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1814db-a8d8-4d28-82fb-bb642771bdb8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:33.093515 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b4028-0424-4cf8-92e1-5204a0eaea55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:33.094109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b4028-0424-4cf8-92e1-5204a0eaea55 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:33.094406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b4028-0424-4cf8-92e1-5204a0eaea55 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:33.094679 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b4028-0424-4cf8-92e1-5204a0eaea55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:33.095387 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b4028-0424-4cf8-92e1-5204a0eaea55 None admin] RBAC: Adding query filter params (role.id=7554b353873942fab506de830188aba0, scope.project.id=d11f33af53374dffa61132642a7770f4, user.id=8cd39d9a583a4d73a5654017be106019) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:33.095688 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b4028-0424-4cf8-92e1-5204a0eaea55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:33.097116 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b4028-0424-4cf8-92e1-5204a0eaea55 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:33.102419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 632/1260] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:58:32 2022] GET /identity/v3/role_assignments?user.id=8cd39d9a583a4d73a5654017be106019&scope.project.id=d11f33af53374dffa61132642a7770f4&role.id=7554b353873942fab506de830188aba0 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:35.124004 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00687aca-812b-4e55-ba78-d76bb942f79d None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:35.124551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00687aca-812b-4e55-ba78-d76bb942f79d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:35.124551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00687aca-812b-4e55-ba78-d76bb942f79d None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:35.125909 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 629/1261] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:58:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:58:35.131207 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb80ec-0de7-4395-92c5-ab5cc7d6cd64 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:35.131609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb80ec-0de7-4395-92c5-ab5cc7d6cd64 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:35.131747 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb80ec-0de7-4395-92c5-ab5cc7d6cd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:35.151007 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0eeb80ec-0de7-4395-92c5-ab5cc7d6cd64 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:58:35.152196 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0eeb80ec-0de7-4395-92c5-ab5cc7d6cd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:35.218246 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 633/1262] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:58:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:58:35.240989 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00687aca-812b-4e55-ba78-d76bb942f79d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:35.252356 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:35.344350 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00687aca-812b-4e55-ba78-d76bb942f79d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:35.378423 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4152b8-57e1-402d-86ce-4bdb2e2178f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:35.379002 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4152b8-57e1-402d-86ce-4bdb2e2178f2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:35.379274 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4152b8-57e1-402d-86ce-4bdb2e2178f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:35.379534 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4152b8-57e1-402d-86ce-4bdb2e2178f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:35.389422 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4152b8-57e1-402d-86ce-4bdb2e2178f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:58:35.394022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-aa4152b8-57e1-402d-86ce-4bdb2e2178f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:58:35.394022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:35.394022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:35.394022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:35.394022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:35.394022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:35.394022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:35.394022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:35.394022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:35.394022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:35.394022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:35.394022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:35.394022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:58:35.394022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:58:35.394022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:58:35.394022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:58:35.395222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:58:35.395222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:58:35.395222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:58:35.395222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:58:35.395222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:35.395222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:35.395222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:35.395222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:35.395222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:35.395222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:35.395222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:35.395222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:35.395222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:35.395222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:35.395222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:35.395222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:35.396320 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:35.396320 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:35.396320 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:35.396320 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:35.396320 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:58:35.396320 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:58:35.396320 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:58:35.396320 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:58:35.396320 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:58:35.396320 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:58:35.396320 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:58:35.396320 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:58:35.396320 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 630/1263] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:58:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:35.402004 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb80ec-0de7-4395-92c5-ab5cc7d6cd64 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:35.420516 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:35.499757 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb80ec-0de7-4395-92c5-ab5cc7d6cd64 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:35.534414 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf43e50-e73a-44c3-a5fc-95ece27fb1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:35.535057 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf43e50-e73a-44c3-a5fc-95ece27fb1bc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:35.535333 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf43e50-e73a-44c3-a5fc-95ece27fb1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:35.535599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf43e50-e73a-44c3-a5fc-95ece27fb1bc None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:35.536256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf43e50-e73a-44c3-a5fc-95ece27fb1bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:35.536551 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf43e50-e73a-44c3-a5fc-95ece27fb1bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:35.537915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf43e50-e73a-44c3-a5fc-95ece27fb1bc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:35.546869 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 634/1264] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:58:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:35.552972 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4152b8-57e1-402d-86ce-4bdb2e2178f2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:35.623444 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:35.657463 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4152b8-57e1-402d-86ce-4bdb2e2178f2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:35.690821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8def0-3d51-4cf6-b9f5-ae8e47cbe1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:35.691436 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8def0-3d51-4cf6-b9f5-ae8e47cbe1cd None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:35.691711 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8def0-3d51-4cf6-b9f5-ae8e47cbe1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:35.691978 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8def0-3d51-4cf6-b9f5-ae8e47cbe1cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:35.702243 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff8def0-3d51-4cf6-b9f5-ae8e47cbe1cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:35.703706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff8def0-3d51-4cf6-b9f5-ae8e47cbe1cd None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:35.717421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-aff8def0-3d51-4cf6-b9f5-ae8e47cbe1cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:58:35.717421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:35.717421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:35.717421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:35.717421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:35.717421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:35.717421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:35.717421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:35.717421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:35.717421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:35.717421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:35.717421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:35.717421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:58:35.717421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:58:35.717421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:58:35.717421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:58:35.718568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:35.718568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:35.718568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:35.718568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:35.718568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:35.718568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:35.718568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:35.718568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:35.718568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:35.718568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:35.718568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:35.718568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:35.718568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:35.718568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:35.718568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:35.718568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:35.718568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:58:35.719843 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:58:35.719843 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:58:35.719843 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:58:35.719843 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 631/1265] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:58:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:35.724498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf43e50-e73a-44c3-a5fc-95ece27fb1bc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:35.809042 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:35.827412 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf43e50-e73a-44c3-a5fc-95ece27fb1bc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:35.860430 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f006000-39a8-4e8c-9be7-577af6572ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:35.861028 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f006000-39a8-4e8c-9be7-577af6572ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:35.861305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f006000-39a8-4e8c-9be7-577af6572ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:35.861588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f006000-39a8-4e8c-9be7-577af6572ab9 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:35.862238 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f006000-39a8-4e8c-9be7-577af6572ab9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:35.862566 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f006000-39a8-4e8c-9be7-577af6572ab9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:35.863933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f006000-39a8-4e8c-9be7-577af6572ab9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:35.875974 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 635/1266] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:58:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:58:35.881760 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8def0-3d51-4cf6-b9f5-ae8e47cbe1cd None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:35.983038 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8def0-3d51-4cf6-b9f5-ae8e47cbe1cd None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:36.006736 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:36.020668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3177b7fa-7912-485f-9fc1-dabb981d3abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:36.022341 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177b7fa-7912-485f-9fc1-dabb981d3abf None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:36.022341 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177b7fa-7912-485f-9fc1-dabb981d3abf None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:36.022341 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177b7fa-7912-485f-9fc1-dabb981d3abf None admin] PATH_INFO: `/v3/users/placement` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:36.031537 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3177b7fa-7912-485f-9fc1-dabb981d3abf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:36.033166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3177b7fa-7912-485f-9fc1-dabb981d3abf None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:36.044792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-3177b7fa-7912-485f-9fc1-dabb981d3abf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 16:58:36.044792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:36.044792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:36.044792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:36.044792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:36.044792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:36.044792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:36.044792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:36.044792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:36.044792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:36.044792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:36.044792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:36.044792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:58:36.044792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:58:36.044792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:58:36.044792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:58:36.045921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:36.045921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:36.045921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:58:36.045921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:58:36.045921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:58:36.045921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:58:36.045921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:36.045921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:36.045921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:36.045921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:36.045921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:36.045921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:36.045921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:36.045921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:36.045921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:36.045921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:36.045921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:36.045921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:36.047261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:36.047261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:36.047261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:58:36.047261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:58:36.047261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:58:36.047261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:58:36.047261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:58:36.047261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:58:36.047261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 16:58:36.047261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:58:36.047261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 632/1267] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:58:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:36.052010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f006000-39a8-4e8c-9be7-577af6572ab9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:36.153290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f006000-39a8-4e8c-9be7-577af6572ab9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:36.186578 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8327280-7e90-483b-bc7a-26da551da08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:36.187176 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8327280-7e90-483b-bc7a-26da551da08d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:36.187487 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8327280-7e90-483b-bc7a-26da551da08d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:36.187765 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8327280-7e90-483b-bc7a-26da551da08d None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:36.188477 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8327280-7e90-483b-bc7a-26da551da08d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:36.188776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8327280-7e90-483b-bc7a-26da551da08d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:36.190191 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8327280-7e90-483b-bc7a-26da551da08d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:36.198818 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:36.207088 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 636/1268] 10.222.0.45 () {56 vars in 1143 bytes} [Wed Aug 10 16:58:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:36.213449 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3177b7fa-7912-485f-9fc1-dabb981d3abf None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:36.314258 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3177b7fa-7912-485f-9fc1-dabb981d3abf None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:36.348844 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db97cd-5de6-47b9-9340-1ba046e0cf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:36.349451 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db97cd-5de6-47b9-9340-1ba046e0cf02 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:36.352239 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db97cd-5de6-47b9-9340-1ba046e0cf02 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:36.352239 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db97cd-5de6-47b9-9340-1ba046e0cf02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:36.360524 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1db97cd-5de6-47b9-9340-1ba046e0cf02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:36.361952 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1db97cd-5de6-47b9-9340-1ba046e0cf02 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:36.375955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-b1db97cd-5de6-47b9-9340-1ba046e0cf02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:58:36.375955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:36.375955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:36.375955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:36.375955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:36.375955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:36.375955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:36.375955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:36.375955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:36.375955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:36.375955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:36.375955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:36.375955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:58:36.375955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:58:36.375955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:58:36.375955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:58:36.377112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:36.377112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:36.377112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:36.377112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:36.377112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:36.377112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:36.377112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:36.377112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:36.377112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:36.377112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:36.377112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:36.377112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:36.377112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:36.377112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:36.377112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:36.377112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:36.377112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:58:36.378289 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:58:36.378289 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:58:36.378289 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:58:36.378289 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 633/1269] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:58:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 16:58:36.383066 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8327280-7e90-483b-bc7a-26da551da08d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:36.481400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8327280-7e90-483b-bc7a-26da551da08d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:36.515103 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a57982-92da-4709-ab45-280be8202815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:36.515699 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a57982-92da-4709-ab45-280be8202815 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:36.515973 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a57982-92da-4709-ab45-280be8202815 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:36.516239 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a57982-92da-4709-ab45-280be8202815 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:36.516863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a57982-92da-4709-ab45-280be8202815 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:36.517157 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a57982-92da-4709-ab45-280be8202815 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:36.518609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a57982-92da-4709-ab45-280be8202815 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:36.530553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 637/1270] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:58:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:36.536379 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db97cd-5de6-47b9-9340-1ba046e0cf02 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:36.568409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:36.638216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db97cd-5de6-47b9-9340-1ba046e0cf02 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:36.674028 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a276f0-bb5c-43ed-935d-ea733cec9b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:36.674690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a276f0-bb5c-43ed-935d-ea733cec9b06 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:36.674964 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a276f0-bb5c-43ed-935d-ea733cec9b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:36.675233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a276f0-bb5c-43ed-935d-ea733cec9b06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:36.682280 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a276f0-bb5c-43ed-935d-ea733cec9b06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:36.683736 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a276f0-bb5c-43ed-935d-ea733cec9b06 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:36.695640 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-39a276f0-bb5c-43ed-935d-ea733cec9b06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:58:36.695640 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:36.695640 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:36.695640 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:36.695640 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:36.695640 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:36.695640 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:36.695640 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:36.695640 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:36.695640 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:36.695640 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:36.695640 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:36.695640 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:58:36.695640 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:58:36.695640 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:58:36.697519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:58:36.697519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:36.697519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:36.697519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:36.697519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:36.697519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:36.697519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:36.697519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:36.697519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:36.697519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:36.697519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:36.697519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:36.697519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:36.697519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:36.697519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:36.697519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:36.697519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:36.698470 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:58:36.698470 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:58:36.698470 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:58:36.698470 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:58:36.698470 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:58:36.698470 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:58:36.698470 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:58:36.698470 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:58:36.698470 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 634/1271] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:58:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:36.703484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a57982-92da-4709-ab45-280be8202815 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:36.762664 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:36.802860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a57982-92da-4709-ab45-280be8202815 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:36.837570 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd682c82-22fd-42d6-bb24-cd0c2bf37bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:36.838413 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd682c82-22fd-42d6-bb24-cd0c2bf37bca None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:36.838695 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd682c82-22fd-42d6-bb24-cd0c2bf37bca None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:36.838965 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd682c82-22fd-42d6-bb24-cd0c2bf37bca None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:36.839727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd682c82-22fd-42d6-bb24-cd0c2bf37bca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:36.839983 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd682c82-22fd-42d6-bb24-cd0c2bf37bca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:36.841416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd682c82-22fd-42d6-bb24-cd0c2bf37bca None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:36.853429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 638/1272] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:58:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:36.859605 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a276f0-bb5c-43ed-935d-ea733cec9b06 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:36.951361 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:36.959835 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a276f0-bb5c-43ed-935d-ea733cec9b06 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:36.992442 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0e699-9a2c-43db-b592-f89e7f74e5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:36.993650 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0e699-9a2c-43db-b592-f89e7f74e5d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:36.994006 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0e699-9a2c-43db-b592-f89e7f74e5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:36.994275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0e699-9a2c-43db-b592-f89e7f74e5d3 None admin] PATH_INFO: `/v3/projects/d11f33af53374dffa61132642a7770f4/users/8cd39d9a583a4d73a5654017be106019/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:36.996225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b0e699-9a2c-43db-b592-f89e7f74e5d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=d11f33af53374dffa61132642a7770f4, user_id=8cd39d9a583a4d73a5654017be106019, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:36.997630 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b0e699-9a2c-43db-b592-f89e7f74e5d3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:37.116301 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 635/1273] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 10 16:58:36 2022] PUT /identity/v3/projects/d11f33af53374dffa61132642a7770f4/users/8cd39d9a583a4d73a5654017be106019/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 16:58:39.100536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef1797-85c8-4353-bd0b-ebd85eb21700 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:39.101093 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef1797-85c8-4353-bd0b-ebd85eb21700 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:39.101093 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef1797-85c8-4353-bd0b-ebd85eb21700 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:39.102596 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 639/1274] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:58:39.107690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa343f-fb76-4e31-9cd5-a3a4a41156a9 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:39.108088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa343f-fb76-4e31-9cd5-a3a4a41156a9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:39.108231 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa343f-fb76-4e31-9cd5-a3a4a41156a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:39.126632 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-71fa343f-fb76-4e31-9cd5-a3a4a41156a9 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:58:39.127810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-71fa343f-fb76-4e31-9cd5-a3a4a41156a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:39.193237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 636/1275] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:58:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:58:39.217514 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef1797-85c8-4353-bd0b-ebd85eb21700 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:39.318926 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef1797-85c8-4353-bd0b-ebd85eb21700 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:39.337378 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:39.354898 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4563bdd2-4f0c-4409-b710-d1898992c6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:39.355462 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563bdd2-4f0c-4409-b710-d1898992c6ba None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:39.355739 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563bdd2-4f0c-4409-b710-d1898992c6ba None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:39.356008 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563bdd2-4f0c-4409-b710-d1898992c6ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:39.366135 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4563bdd2-4f0c-4409-b710-d1898992c6ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 16:58:39.371558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4563bdd2-4f0c-4409-b710-d1898992c6ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:58:39.371558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:39.371558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:39.371558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:39.371558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:39.371558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:39.371558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:39.371558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:39.371558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:39.371558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:39.371558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:39.371558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:39.371558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 16:58:39.371558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 16:58:39.371558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 16:58:39.371558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 16:58:39.376593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 16:58:39.376593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 16:58:39.376593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 16:58:39.376593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 16:58:39.376593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:39.376593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:39.376593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:39.376593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:39.376593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:39.376593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:39.376593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:39.376593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:39.376593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:39.376593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:39.376593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:39.376593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:39.377728 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:39.377728 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:39.377728 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:39.377728 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:39.377728 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 16:58:39.377728 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 16:58:39.377728 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 16:58:39.377728 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 16:58:39.377728 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 16:58:39.377728 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 16:58:39.377728 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 16:58:39.377728 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:58:39.377728 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 640/1276] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 10 16:58:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:39.378944 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa343f-fb76-4e31-9cd5-a3a4a41156a9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:39.458125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:39.475612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa343f-fb76-4e31-9cd5-a3a4a41156a9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:39.510742 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb4e99-0469-4b1d-8832-a2885b75440b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:39.511339 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb4e99-0469-4b1d-8832-a2885b75440b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:39.511612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb4e99-0469-4b1d-8832-a2885b75440b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:39.511879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb4e99-0469-4b1d-8832-a2885b75440b None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:39.512536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb4e99-0469-4b1d-8832-a2885b75440b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:39.512851 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb4e99-0469-4b1d-8832-a2885b75440b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:39.514264 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb4e99-0469-4b1d-8832-a2885b75440b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:39.522280 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 637/1277] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 10 16:58:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:39.528072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4563bdd2-4f0c-4409-b710-d1898992c6ba None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:39.640725 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4563bdd2-4f0c-4409-b710-d1898992c6ba None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:39.675321 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72d74a-4225-49e5-a466-439e2f9bb8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:39.675984 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72d74a-4225-49e5-a466-439e2f9bb8bf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:39.676266 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72d74a-4225-49e5-a466-439e2f9bb8bf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:39.676541 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72d74a-4225-49e5-a466-439e2f9bb8bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:39.694669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72d74a-4225-49e5-a466-439e2f9bb8bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:39.696156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72d74a-4225-49e5-a466-439e2f9bb8bf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:39.712837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-dc72d74a-4225-49e5-a466-439e2f9bb8bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:58:39.712837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:39.712837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:39.712837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:39.712837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:39.712837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:39.712837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:39.712837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:39.712837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:39.712837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:39.712837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:39.712837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:39.712837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:58:39.712837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:58:39.712837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:58:39.712837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:58:39.714044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:39.714044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:39.714044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:39.714044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:39.714044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:39.714044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:39.714044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:39.714044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:39.714044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:39.714044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:39.714044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:39.714044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:39.714044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:39.714044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:39.714044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:39.714044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:39.714044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:58:39.715259 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:58:39.715259 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:58:39.715259 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:58:39.715259 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 641/1278] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:58:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:39.719928 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb4e99-0469-4b1d-8832-a2885b75440b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:39.819882 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb4e99-0469-4b1d-8832-a2885b75440b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:39.853233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ce5a0-4b15-4f14-bd80-57a7e7e503e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:39.853820 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ce5a0-4b15-4f14-bd80-57a7e7e503e7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:39.854088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ce5a0-4b15-4f14-bd80-57a7e7e503e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:39.854373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ce5a0-4b15-4f14-bd80-57a7e7e503e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:39.855000 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ce5a0-4b15-4f14-bd80-57a7e7e503e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:39.855294 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ce5a0-4b15-4f14-bd80-57a7e7e503e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:39.856639 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ce5a0-4b15-4f14-bd80-57a7e7e503e7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:39.869822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 638/1279] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:39.873824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72d74a-4225-49e5-a466-439e2f9bb8bf None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:39.904660 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:39.971129 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72d74a-4225-49e5-a466-439e2f9bb8bf None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:40.009041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63daa1f2-89a2-4caf-bf34-b59d88c0a638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:40.009670 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63daa1f2-89a2-4caf-bf34-b59d88c0a638 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:40.009943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63daa1f2-89a2-4caf-bf34-b59d88c0a638 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:40.010209 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63daa1f2-89a2-4caf-bf34-b59d88c0a638 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:40.017827 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63daa1f2-89a2-4caf-bf34-b59d88c0a638 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:40.020843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63daa1f2-89a2-4caf-bf34-b59d88c0a638 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:40.033137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-63daa1f2-89a2-4caf-bf34-b59d88c0a638 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 16:58:40.033137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:40.033137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:40.033137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:40.033137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:40.033137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:40.033137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:40.033137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:40.033137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:40.033137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:40.033137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:40.033137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:40.033137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 16:58:40.033137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 16:58:40.033137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 16:58:40.033137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 16:58:40.034265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:40.034265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:40.034265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 16:58:40.034265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:58:40.034265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 16:58:40.034265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 16:58:40.034265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:40.034265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:40.034265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:40.034265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:40.034265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:40.034265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:40.034265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:40.034265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:40.034265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:40.034265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:40.034265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:40.034265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:40.035280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:40.035280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:40.035280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 16:58:40.035280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 16:58:40.035280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 16:58:40.035280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 16:58:40.035280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 16:58:40.035280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 16:58:40.035280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 16:58:40.035280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:58:40.035280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 642/1280] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:58:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:40.040923 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ce5a0-4b15-4f14-bd80-57a7e7e503e7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:40.044655 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:40.136671 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ce5a0-4b15-4f14-bd80-57a7e7e503e7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:40.170809 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8b628-eda9-470f-9b12-ef4cfeb81cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:40.171404 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8b628-eda9-470f-9b12-ef4cfeb81cbf None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:40.171680 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8b628-eda9-470f-9b12-ef4cfeb81cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:40.171947 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8b628-eda9-470f-9b12-ef4cfeb81cbf None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:40.172661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8b628-eda9-470f-9b12-ef4cfeb81cbf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:40.172959 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8b628-eda9-470f-9b12-ef4cfeb81cbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:40.174417 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8b628-eda9-470f-9b12-ef4cfeb81cbf None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:40.192005 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 639/1281] 10.222.0.45 () {56 vars in 1143 bytes} [Wed Aug 10 16:58:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:40.198460 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63daa1f2-89a2-4caf-bf34-b59d88c0a638 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:40.287462 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:40.296926 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63daa1f2-89a2-4caf-bf34-b59d88c0a638 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:40.329996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914304a-6005-49ed-9eda-57b9b044dc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:40.330602 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914304a-6005-49ed-9eda-57b9b044dc4c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:40.330880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914304a-6005-49ed-9eda-57b9b044dc4c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:40.331146 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914304a-6005-49ed-9eda-57b9b044dc4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:40.397677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6914304a-6005-49ed-9eda-57b9b044dc4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:40.399728 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6914304a-6005-49ed-9eda-57b9b044dc4c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:40.494739 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:40.505013 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 643/1282] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 10 16:58:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 16:58:40.511394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8b628-eda9-470f-9b12-ef4cfeb81cbf None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:40.570368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:40.608030 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8b628-eda9-470f-9b12-ef4cfeb81cbf None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:40.642111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd8423-087a-4709-9e9e-37409de26368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:40.642781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd8423-087a-4709-9e9e-37409de26368 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:40.643078 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd8423-087a-4709-9e9e-37409de26368 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:40.643392 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd8423-087a-4709-9e9e-37409de26368 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:40.653443 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecd8423-087a-4709-9e9e-37409de26368 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:40.654863 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecd8423-087a-4709-9e9e-37409de26368 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:40.668970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-eecd8423-087a-4709-9e9e-37409de26368 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:58:40.668970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:40.668970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:40.668970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:40.668970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:40.668970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:40.668970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:40.668970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:40.668970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:40.668970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:40.668970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:40.668970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:40.668970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 16:58:40.668970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 16:58:40.668970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 16:58:40.668970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 16:58:40.670129 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:40.670129 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:40.670129 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:40.670129 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:40.670129 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:40.670129 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:40.670129 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:40.670129 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:40.670129 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:40.670129 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:40.670129 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:40.670129 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:40.670129 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:40.670129 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:40.670129 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:40.670129 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:40.670129 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 16:58:40.671111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 16:58:40.671111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 16:58:40.671111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:58:40.674426 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 640/1283] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 16:58:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:40.675935 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914304a-6005-49ed-9eda-57b9b044dc4c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:40.778720 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914304a-6005-49ed-9eda-57b9b044dc4c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:40.812142 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ab344-aa47-409f-89f1-e394d6899df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:40.812868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ab344-aa47-409f-89f1-e394d6899df8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:40.813142 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ab344-aa47-409f-89f1-e394d6899df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:40.813409 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ab344-aa47-409f-89f1-e394d6899df8 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:40.814040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ab344-aa47-409f-89f1-e394d6899df8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:40.814335 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ab344-aa47-409f-89f1-e394d6899df8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:40.815861 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ab344-aa47-409f-89f1-e394d6899df8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:40.827429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 644/1284] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 10 16:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:40.833122 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd8423-087a-4709-9e9e-37409de26368 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:40.933628 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd8423-087a-4709-9e9e-37409de26368 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:40.947796 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:40.968424 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3571c16-96ed-447a-a441-f62d9f40a4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:40.969044 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3571c16-96ed-447a-a441-f62d9f40a4f1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:40.969321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3571c16-96ed-447a-a441-f62d9f40a4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:40.969591 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3571c16-96ed-447a-a441-f62d9f40a4f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:40.976218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3571c16-96ed-447a-a441-f62d9f40a4f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:40.977880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3571c16-96ed-447a-a441-f62d9f40a4f1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:40.988810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-b3571c16-96ed-447a-a441-f62d9f40a4f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:58:40.988810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:40.988810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:40.988810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:40.988810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:40.988810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:40.988810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:40.988810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:40.988810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:40.988810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:40.988810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:40.988810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:40.988810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 16:58:40.988810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 16:58:40.988810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 16:58:40.989771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 16:58:40.989771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:40.989771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:40.989771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:40.989771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:40.989771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:40.989771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:40.989771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:40.989771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:40.989771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:40.989771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:40.989771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:40.989771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:40.989771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:40.989771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:40.989771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:40.989771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:40.990777 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 16:58:40.990777 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 16:58:40.990777 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 16:58:40.990777 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 16:58:40.990777 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 16:58:40.990777 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 16:58:40.990777 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 16:58:40.990777 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 10 16:58:40.990777 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 641/1285] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 10 16:58:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:40.995967 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ab344-aa47-409f-89f1-e394d6899df8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:41.047358 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:41.096130 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ab344-aa47-409f-89f1-e394d6899df8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:41.128383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f7916-660b-4033-b7dc-84d200877860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:41.129020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f7916-660b-4033-b7dc-84d200877860 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:41.129295 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f7916-660b-4033-b7dc-84d200877860 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:41.129563 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f7916-660b-4033-b7dc-84d200877860 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:41.130273 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f7916-660b-4033-b7dc-84d200877860 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:41.130804 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f7916-660b-4033-b7dc-84d200877860 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:41.132184 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f7916-660b-4033-b7dc-84d200877860 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:41.144273 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 645/1286] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 10 16:58:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:41.151263 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3571c16-96ed-447a-a441-f62d9f40a4f1 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:41.254519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3571c16-96ed-447a-a441-f62d9f40a4f1 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:41.290498 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d2b88-f271-431e-99b8-28347d6011de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:41.291125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d2b88-f271-431e-99b8-28347d6011de None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:41.291402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d2b88-f271-431e-99b8-28347d6011de None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:41.291675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d2b88-f271-431e-99b8-28347d6011de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:41.292400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428d2b88-f271-431e-99b8-28347d6011de None admin] RBAC: Adding query filter params (role.id=7554b353873942fab506de830188aba0, scope.project.id=d11f33af53374dffa61132642a7770f4, user.id=8cd39d9a583a4d73a5654017be106019) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:41.292688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428d2b88-f271-431e-99b8-28347d6011de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:41.294114 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428d2b88-f271-431e-99b8-28347d6011de None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:41.299869 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 642/1287] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 10 16:58:41 2022] GET /identity/v3/role_assignments?user.id=8cd39d9a583a4d73a5654017be106019&scope.project.id=d11f33af53374dffa61132642a7770f4&role.id=7554b353873942fab506de830188aba0 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:43.354443 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61388cfa-71ef-4aa3-ab0e-363c1490db81 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:43.355041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61388cfa-71ef-4aa3-ab0e-363c1490db81 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:43.355041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61388cfa-71ef-4aa3-ab0e-363c1490db81 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:43.356615 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 646/1288] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:58:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:58:43.361837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9531b1-75d0-4cb3-9404-aad8df2fcff9 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:43.362246 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9531b1-75d0-4cb3-9404-aad8df2fcff9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:43.362408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9531b1-75d0-4cb3-9404-aad8df2fcff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:43.380646 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5e9531b1-75d0-4cb3-9404-aad8df2fcff9 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:58:43.381842 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5e9531b1-75d0-4cb3-9404-aad8df2fcff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:43.411969 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:43.448956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 643/1289] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:58:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 16:58:43.470763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61388cfa-71ef-4aa3-ab0e-363c1490db81 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:43.574982 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61388cfa-71ef-4aa3-ab0e-363c1490db81 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:43.585543 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:43.607804 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40abcf33-1859-4bf9-a5aa-76ea55443da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:43.608363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abcf33-1859-4bf9-a5aa-76ea55443da6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:43.609080 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abcf33-1859-4bf9-a5aa-76ea55443da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:43.609080 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abcf33-1859-4bf9-a5aa-76ea55443da6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:43.609440 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40abcf33-1859-4bf9-a5aa-76ea55443da6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:43.610835 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40abcf33-1859-4bf9-a5aa-76ea55443da6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:43.620003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-40abcf33-1859-4bf9-a5aa-76ea55443da6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 16:58:43.620003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:43.620003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:58:43.620003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:58:43.620003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:58:43.620003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:58:43.620003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:58:43.620003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:58:43.620003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 16:58:43.620003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:58:43.620003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:58:43.620003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:58:43.620003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:43.620003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:43.620003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:43.620003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:43.620003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:43.621282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:43.621282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:43.621282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:43.621282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:43.621282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:43.621282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:43.621282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:58:43.621282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:58:43.621282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:58:43.621282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:58:43.621282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:43.621282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:43.621282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:43.621282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:43.621282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:43.621282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:43.622277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:43.622277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:43.622277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:43.622277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:43.622277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:43.622277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:43.622277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:43.622277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:43.622277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:43.622277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:43.622277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:58:43.622277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:58:43.622277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 16:58:43.622277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:58:43.622277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 647/1290] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 10 16:58:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:43.627465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9531b1-75d0-4cb3-9404-aad8df2fcff9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:43.727099 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9531b1-75d0-4cb3-9404-aad8df2fcff9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:43.764205 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fac5e0-5951-4ffd-bdae-d9560be45d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:43.764804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fac5e0-5951-4ffd-bdae-d9560be45d22 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:43.765073 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fac5e0-5951-4ffd-bdae-d9560be45d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:43.765336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fac5e0-5951-4ffd-bdae-d9560be45d22 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:43.765951 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fac5e0-5951-4ffd-bdae-d9560be45d22 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:43.766245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fac5e0-5951-4ffd-bdae-d9560be45d22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:43.767649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fac5e0-5951-4ffd-bdae-d9560be45d22 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:43.772834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 644/1291] 10.222.0.45 () {56 vars in 1116 bytes} [Wed Aug 10 16:58:43 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 16:58:43.779136 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40abcf33-1859-4bf9-a5aa-76ea55443da6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:43.879702 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40abcf33-1859-4bf9-a5aa-76ea55443da6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:43.913562 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11588491-1d25-48b0-86d4-e20743ea3d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:43.914160 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11588491-1d25-48b0-86d4-e20743ea3d56 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:43.914478 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11588491-1d25-48b0-86d4-e20743ea3d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:43.915372 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11588491-1d25-48b0-86d4-e20743ea3d56 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:43.915991 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11588491-1d25-48b0-86d4-e20743ea3d56 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:43.916282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11588491-1d25-48b0-86d4-e20743ea3d56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:43.917639 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11588491-1d25-48b0-86d4-e20743ea3d56 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:43.921982 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 648/1292] 10.222.0.45 () {56 vars in 1116 bytes} [Wed Aug 10 16:58:43 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:45.963874 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b337d-0cf4-41f7-a3cb-f633235cf08e None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:45.964413 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b337d-0cf4-41f7-a3cb-f633235cf08e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:45.964704 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b337d-0cf4-41f7-a3cb-f633235cf08e None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:45.967230 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 645/1293] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:58:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:58:45.972713 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226f6c2-986f-4d80-8d6b-5185b3168fa6 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:45.975115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226f6c2-986f-4d80-8d6b-5185b3168fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:45.975115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226f6c2-986f-4d80-8d6b-5185b3168fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:45.991956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7226f6c2-986f-4d80-8d6b-5185b3168fa6 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:58:45.993229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7226f6c2-986f-4d80-8d6b-5185b3168fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:46.023520 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:46.061695 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 649/1294] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:58:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:58:46.086091 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b337d-0cf4-41f7-a3cb-f633235cf08e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:46.119231 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:46.190936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b337d-0cf4-41f7-a3cb-f633235cf08e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:46.228158 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fafa57-f375-4a26-bc60-705bb3be7661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:46.228666 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fafa57-f375-4a26-bc60-705bb3be7661 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:46.228935 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fafa57-f375-4a26-bc60-705bb3be7661 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:46.229195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fafa57-f375-4a26-bc60-705bb3be7661 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:46.229832 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fafa57-f375-4a26-bc60-705bb3be7661 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:46.231210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fafa57-f375-4a26-bc60-705bb3be7661 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:46.310292 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 646/1295] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 10 16:58:46 2022] POST /identity/v3/services => generated 246 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:58:48.415506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48524e1-8621-4733-822b-05e5d6486d4c None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:48.416063 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48524e1-8621-4733-822b-05e5d6486d4c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:48.416063 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48524e1-8621-4733-822b-05e5d6486d4c None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:48.417413 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 650/1296] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:58:48.422827 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a7ec8-893a-4495-8461-b5a1f0be5aab None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:48.423208 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a7ec8-893a-4495-8461-b5a1f0be5aab None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:48.423372 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a7ec8-893a-4495-8461-b5a1f0be5aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:48.441308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-867a7ec8-893a-4495-8461-b5a1f0be5aab None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:58:48.442569 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-867a7ec8-893a-4495-8461-b5a1f0be5aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:48.514281 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 647/1297] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:58:48 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:58:48.537904 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48524e1-8621-4733-822b-05e5d6486d4c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:48.638810 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48524e1-8621-4733-822b-05e5d6486d4c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:48.677328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a846bb9f-2474-4525-8032-744ff67deb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:48.677328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846bb9f-2474-4525-8032-744ff67deb21 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:48.677328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846bb9f-2474-4525-8032-744ff67deb21 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:48.677328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846bb9f-2474-4525-8032-744ff67deb21 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:48.677742 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a846bb9f-2474-4525-8032-744ff67deb21 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:48.677861 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a846bb9f-2474-4525-8032-744ff67deb21 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:48.687098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-a846bb9f-2474-4525-8032-744ff67deb21 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 16:58:48.687098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:48.687098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:58:48.687098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:58:48.687098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:58:48.687098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:58:48.687098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:58:48.687098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:58:48.687098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 16:58:48.687098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:58:48.687098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:58:48.687098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:58:48.687098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:48.687098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:48.687098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:48.687098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:48.687098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:48.688301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:48.688301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:48.688301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:48.688301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:48.688301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:48.688301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:48.688301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:58:48.688301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:58:48.688301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:58:48.688301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:58:48.688301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:48.688301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:48.688301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:48.688301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:48.688301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:48.688301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:48.689371 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:48.689371 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:48.689371 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:48.689371 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:48.689371 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:48.689371 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:48.689371 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:48.689371 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:48.689371 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:48.689371 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:48.689371 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:58:48.689371 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:58:48.689371 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 16:58:48.689371 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:58:48.689371 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 651/1298] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 10 16:58:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:48.694727 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a7ec8-893a-4495-8461-b5a1f0be5aab None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:48.713213 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:48.798675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a7ec8-893a-4495-8461-b5a1f0be5aab None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:48.837843 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deade28e-a9a0-4171-b771-01c728544eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:48.837843 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deade28e-a9a0-4171-b771-01c728544eac None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:48.837843 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deade28e-a9a0-4171-b771-01c728544eac None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:48.837843 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deade28e-a9a0-4171-b771-01c728544eac None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:48.838329 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deade28e-a9a0-4171-b771-01c728544eac None admin] RBAC: Adding query filter params (name=placement) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:48.838329 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deade28e-a9a0-4171-b771-01c728544eac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:48.838815 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deade28e-a9a0-4171-b771-01c728544eac None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:48.844010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 648/1299] 10.222.0.45 () {56 vars in 1116 bytes} [Wed Aug 10 16:58:48 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:48.850173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a846bb9f-2474-4525-8032-744ff67deb21 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:48.882019 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:48.950526 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a846bb9f-2474-4525-8032-744ff67deb21 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:48.984965 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2cd78f-8920-408b-b393-d9663e9ad1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:48.985564 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2cd78f-8920-408b-b393-d9663e9ad1ba None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:48.985843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2cd78f-8920-408b-b393-d9663e9ad1ba None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:48.986111 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2cd78f-8920-408b-b393-d9663e9ad1ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:48.986828 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2cd78f-8920-408b-b393-d9663e9ad1ba None admin] RBAC: Adding query filter params (interface=public, service_id=44d3253793c04ebb9bccee2d77af824e, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:48.987132 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2cd78f-8920-408b-b393-d9663e9ad1ba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:48.988493 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2cd78f-8920-408b-b393-d9663e9ad1ba None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:48.993931 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 652/1300] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Aug 10 16:58:48 2022] GET /identity/v3/endpoints?service_id=44d3253793c04ebb9bccee2d77af824e&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:48.999635 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deade28e-a9a0-4171-b771-01c728544eac None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:49.079556 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:49.097069 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deade28e-a9a0-4171-b771-01c728544eac None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:49.130519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82287a-a244-41e2-8a34-1f917593139e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:49.131117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82287a-a244-41e2-8a34-1f917593139e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:49.131395 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82287a-a244-41e2-8a34-1f917593139e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:49.131667 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82287a-a244-41e2-8a34-1f917593139e None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:49.132266 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff82287a-a244-41e2-8a34-1f917593139e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:49.133899 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff82287a-a244-41e2-8a34-1f917593139e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:49.140773 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 649/1301] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 10 16:58:48 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 16:58:51.202803 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94737c-2441-4e27-b992-60344ad1570d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:51.203353 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94737c-2441-4e27-b992-60344ad1570d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:51.203353 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94737c-2441-4e27-b992-60344ad1570d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:51.204669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 653/1302] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 16:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 16:58:51.209734 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836be92e-07e1-4726-a823-f2e83cae2f20 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:51.210119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836be92e-07e1-4726-a823-f2e83cae2f20 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:51.210275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836be92e-07e1-4726-a823-f2e83cae2f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:51.229376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-836be92e-07e1-4726-a823-f2e83cae2f20 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 16:58:51.231012 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-836be92e-07e1-4726-a823-f2e83cae2f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 16:58:51.298753 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 650/1303] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 16:58:51 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 16:58:51.322467 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94737c-2441-4e27-b992-60344ad1570d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:51.428291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:51.435279 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94737c-2441-4e27-b992-60344ad1570d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:51.470742 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5714d2-44e7-4452-9275-98194f07310a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:51.471289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5714d2-44e7-4452-9275-98194f07310a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:51.471559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5714d2-44e7-4452-9275-98194f07310a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:51.471828 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5714d2-44e7-4452-9275-98194f07310a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:51.472400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5714d2-44e7-4452-9275-98194f07310a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:51.473763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5714d2-44e7-4452-9275-98194f07310a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:51.482628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-cf5714d2-44e7-4452-9275-98194f07310a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 16:58:51.482628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:51.482628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 16:58:51.482628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 16:58:51.482628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 16:58:51.482628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 16:58:51.482628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 16:58:51.482628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:58:51.482628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 16:58:51.482628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:58:51.482628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 16:58:51.482628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:58:51.482628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 16:58:51.482628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 16:58:51.482628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 16:58:51.482628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 16:58:51.482628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 16:58:51.483628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 16:58:51.483628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 16:58:51.483628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 16:58:51.483628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 16:58:51.483628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 16:58:51.483628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 16:58:51.483628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 16:58:51.483628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 16:58:51.483628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 16:58:51.483628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 16:58:51.483628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 16:58:51.483628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 16:58:51.483628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 16:58:51.483628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 16:58:51.483628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 16:58:51.483628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 16:58:51.484521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 16:58:51.484521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Aug 10 16:58:51.484521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 16:58:51.484521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Aug 10 16:58:51.484521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 16:58:51.484521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 16:58:51.484521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 16:58:51.484521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Aug 10 16:58:51.484521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 16:58:51.484521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Aug 10 16:58:51.484521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 16:58:51.484521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 16:58:51.484521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 16:58:51.484521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Aug 10 16:58:51.484521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 654/1304] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 10 16:58:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 16:58:51.489996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836be92e-07e1-4726-a823-f2e83cae2f20 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:51.567821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:58:51.590896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836be92e-07e1-4726-a823-f2e83cae2f20 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:51.624525 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf7edf-f0c3-42b3-a330-e373d33b2b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:51.625123 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf7edf-f0c3-42b3-a330-e373d33b2b80 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:51.625394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf7edf-f0c3-42b3-a330-e373d33b2b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:51.625656 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf7edf-f0c3-42b3-a330-e373d33b2b80 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:51.626272 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf7edf-f0c3-42b3-a330-e373d33b2b80 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 16:58:51.626599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf7edf-f0c3-42b3-a330-e373d33b2b80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:51.628044 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf7edf-f0c3-42b3-a330-e373d33b2b80 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:51.632957 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 651/1305] 10.222.0.45 () {56 vars in 1116 bytes} [Wed Aug 10 16:58:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 16:58:51.639524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5714d2-44e7-4452-9275-98194f07310a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 16:58:51.739446 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5714d2-44e7-4452-9275-98194f07310a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 16:58:51.772648 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ea639-861c-4c99-ae3e-208215b2129e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 16:58:51.773235 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ea639-861c-4c99-ae3e-208215b2129e None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 16:58:51.773510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ea639-861c-4c99-ae3e-208215b2129e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 16:58:51.773807 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ea639-861c-4c99-ae3e-208215b2129e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 16:58:51.774479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ea639-861c-4c99-ae3e-208215b2129e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 16:58:51.775828 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ea639-861c-4c99-ae3e-208215b2129e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 16:58:51.897572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 655/1306] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 16:58:51 2022] POST /identity/v3/endpoints => generated 331 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 16:59:01.675250 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=30 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 16:59:01.892244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:05:54.477407 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c5d66-fed4-4a40-b5af-15c9aa5d3623 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:05:54.477945 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c5d66-fed4-4a40-b5af-15c9aa5d3623 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:05:54.477945 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c5d66-fed4-4a40-b5af-15c9aa5d3623 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:05:54.479226 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 652/1307] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:05:54.485647 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000de64-7be9-41b2-8204-d1946765c0ee None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:05:54.486560 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000de64-7be9-41b2-8204-d1946765c0ee None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:05:54.487047 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000de64-7be9-41b2-8204-d1946765c0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:05:54.565876 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4000de64-7be9-41b2-8204-d1946765c0ee None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:05:54.567462 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4000de64-7be9-41b2-8204-d1946765c0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:05:54.710892 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 656/1308] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:05:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:05:54.774215 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d8112-adbb-4c89-b509-3e7867574c62 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:05:54.775607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d8112-adbb-4c89-b509-3e7867574c62 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:05:54.777022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d8112-adbb-4c89-b509-3e7867574c62 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:05:54.782272 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 653/1309] 10.222.0.45 () {54 vars in 965 bytes} [Wed Aug 10 17:05:54 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:05:54.790682 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870d413-1074-42bc-aebb-577df168c2e1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:05:54.792245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870d413-1074-42bc-aebb-577df168c2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:05:54.793588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870d413-1074-42bc-aebb-577df168c2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:05:54.921758 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f870d413-1074-42bc-aebb-577df168c2e1 None None] MFA Rules not processed for user `e1534cfef9e9428bb2e010fff200bc19`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:05:54.923225 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f870d413-1074-42bc-aebb-577df168c2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:05:54.952976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:05:55.041963 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 657/1310] 10.222.0.45 () {58 vars in 1062 bytes} [Wed Aug 10 17:05:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:05:55.062653 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d8112-adbb-4c89-b509-3e7867574c62 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:05:55.064792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9b9d8112-adbb-4c89-b509-3e7867574c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:05:55.202152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d8112-adbb-4c89-b509-3e7867574c62 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:05:55.294455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a087f3f-be90-40a7-a9af-6e5b709731f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:05:55.295084 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a087f3f-be90-40a7-a9af-6e5b709731f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:05:55.295495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a087f3f-be90-40a7-a9af-6e5b709731f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:05:55.295769 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a087f3f-be90-40a7-a9af-6e5b709731f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:05:55.297455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5a087f3f-be90-40a7-a9af-6e5b709731f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:05:55.367499 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:05:55.437116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a087f3f-be90-40a7-a9af-6e5b709731f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:05:55.440131 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a087f3f-be90-40a7-a9af-6e5b709731f8 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:05:55.473402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 654/1311] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:05:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:05:57.839134 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209c4bc-5c58-44bf-a1c1-3f0d11b3c4ca None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:05:57.839710 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209c4bc-5c58-44bf-a1c1-3f0d11b3c4ca None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:05:57.839710 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209c4bc-5c58-44bf-a1c1-3f0d11b3c4ca None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:05:57.841112 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 658/1312] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:05:57.848029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ad95c-29c4-43b4-b730-a556f8e54add None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:05:57.848595 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ad95c-29c4-43b4-b730-a556f8e54add None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:05:57.848595 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ad95c-29c4-43b4-b730-a556f8e54add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:05:57.937867 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-440ad95c-29c4-43b4-b730-a556f8e54add None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:05:57.939130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-440ad95c-29c4-43b4-b730-a556f8e54add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:05:58.043061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 655/1313] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:05:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:05:58.082037 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209c4bc-5c58-44bf-a1c1-3f0d11b3c4ca None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:05:58.083646 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0209c4bc-5c58-44bf-a1c1-3f0d11b3c4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:05:58.199545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209c4bc-5c58-44bf-a1c1-3f0d11b3c4ca None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:05:58.277159 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:05:58.279579 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d298f-7a12-4f55-953d-f873dfa227ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:05:58.280184 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d298f-7a12-4f55-953d-f873dfa227ce service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:05:58.280459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d298f-7a12-4f55-953d-f873dfa227ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:05:58.280725 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d298f-7a12-4f55-953d-f873dfa227ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:05:58.282381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-760d298f-7a12-4f55-953d-f873dfa227ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:05:58.393938 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d298f-7a12-4f55-953d-f873dfa227ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:05:58.395344 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d298f-7a12-4f55-953d-f873dfa227ce service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:05:58.414093 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 659/1314] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:05:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:06:01.025526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82868c2-0ff0-4110-8879-ab409b027515 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:01.026079 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82868c2-0ff0-4110-8879-ab409b027515 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:01.026079 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82868c2-0ff0-4110-8879-ab409b027515 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:01.027367 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 656/1315] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:06:01.033144 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9111d6-c3dd-4b0c-94f5-9bf98a23e82f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:01.033539 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9111d6-c3dd-4b0c-94f5-9bf98a23e82f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:01.033709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9111d6-c3dd-4b0c-94f5-9bf98a23e82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:01.099461 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ea9111d6-c3dd-4b0c-94f5-9bf98a23e82f None None] MFA Rules not processed for user `b98eacd3b6474daabcffaf3ef31aea2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:01.100644 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ea9111d6-c3dd-4b0c-94f5-9bf98a23e82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:01.150778 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:01.194468 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 660/1316] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:06:01 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:01.248339 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22609f-5e17-4ba1-8755-cddcadfff918 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:01.248620 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22609f-5e17-4ba1-8755-cddcadfff918 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:01.248936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22609f-5e17-4ba1-8755-cddcadfff918 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:01.250294 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 657/1317] 10.222.0.45 () {54 vars in 965 bytes} [Wed Aug 10 17:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:06:01.256238 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea009f-d375-4424-b328-58faa629516b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:01.256520 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea009f-d375-4424-b328-58faa629516b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:01.256790 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea009f-d375-4424-b328-58faa629516b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:01.330924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-beea009f-d375-4424-b328-58faa629516b None None] MFA Rules not processed for user `e1534cfef9e9428bb2e010fff200bc19`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:01.332040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-beea009f-d375-4424-b328-58faa629516b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:01.407689 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 661/1318] 10.222.0.45 () {58 vars in 1062 bytes} [Wed Aug 10 17:06:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:01.416718 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22609f-5e17-4ba1-8755-cddcadfff918 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:01.418296 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cc22609f-5e17-4ba1-8755-cddcadfff918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:01.428836 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:01.521829 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22609f-5e17-4ba1-8755-cddcadfff918 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:01.596543 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2c04a-f196-44e6-90eb-4c81f2b19b10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:01.597110 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2c04a-f196-44e6-90eb-4c81f2b19b10 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:01.597397 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2c04a-f196-44e6-90eb-4c81f2b19b10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:01.597665 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2c04a-f196-44e6-90eb-4c81f2b19b10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:01.599291 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-31e2c04a-f196-44e6-90eb-4c81f2b19b10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:01.663699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:01.710592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2c04a-f196-44e6-90eb-4c81f2b19b10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:01.712023 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2c04a-f196-44e6-90eb-4c81f2b19b10 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:01.730654 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 658/1319] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:01 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:06:05.325546 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f06f15-0afd-42f3-bf0e-6c4b5418e2a9 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:05.326167 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f06f15-0afd-42f3-bf0e-6c4b5418e2a9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:05.326532 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f06f15-0afd-42f3-bf0e-6c4b5418e2a9 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:05.327905 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 662/1320] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:06:05.333133 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfa078-98ee-4e03-a769-8c9a471dde6e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:05.333557 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfa078-98ee-4e03-a769-8c9a471dde6e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:05.333652 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfa078-98ee-4e03-a769-8c9a471dde6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:05.399037 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-96cfa078-98ee-4e03-a769-8c9a471dde6e None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:05.400248 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-96cfa078-98ee-4e03-a769-8c9a471dde6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:05.497256 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 659/1321] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:06:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:05.537776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f06f15-0afd-42f3-bf0e-6c4b5418e2a9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:05.539351 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f9f06f15-0afd-42f3-bf0e-6c4b5418e2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:05.590971 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:05.646996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f06f15-0afd-42f3-bf0e-6c4b5418e2a9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:05.719117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea7cef-5241-4fa6-ba06-17b8c42f10b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:05.719710 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea7cef-5241-4fa6-ba06-17b8c42f10b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:05.719982 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea7cef-5241-4fa6-ba06-17b8c42f10b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:05.720246 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea7cef-5241-4fa6-ba06-17b8c42f10b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:05.721818 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8fea7cef-5241-4fa6-ba06-17b8c42f10b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:05.818108 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:05.831366 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea7cef-5241-4fa6-ba06-17b8c42f10b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:05.832863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea7cef-5241-4fa6-ba06-17b8c42f10b6 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:05.851432 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 663/1322] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:06:08.977671 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400fd42f-743d-4f9e-9c97-900ac07673e2 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:08.978245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400fd42f-743d-4f9e-9c97-900ac07673e2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:08.978245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400fd42f-743d-4f9e-9c97-900ac07673e2 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:08.979714 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 660/1323] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:06:08.985275 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480599ed-de45-45f4-b5fc-474a35f2d39e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:08.985676 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480599ed-de45-45f4-b5fc-474a35f2d39e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:08.985897 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480599ed-de45-45f4-b5fc-474a35f2d39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:09.054635 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-480599ed-de45-45f4-b5fc-474a35f2d39e None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:09.055836 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-480599ed-de45-45f4-b5fc-474a35f2d39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:09.156722 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 664/1324] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:09.194049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400fd42f-743d-4f9e-9c97-900ac07673e2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:09.195740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-400fd42f-743d-4f9e-9c97-900ac07673e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:09.214710 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:09.306973 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400fd42f-743d-4f9e-9c97-900ac07673e2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:09.390961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ae28b-b417-4c71-8956-89306bafc33e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:09.391606 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ae28b-b417-4c71-8956-89306bafc33e service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:09.391882 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ae28b-b417-4c71-8956-89306bafc33e service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:09.392149 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ae28b-b417-4c71-8956-89306bafc33e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:09.394141 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-af7ae28b-b417-4c71-8956-89306bafc33e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:09.471454 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:09.508692 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ae28b-b417-4c71-8956-89306bafc33e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:09.510124 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ae28b-b417-4c71-8956-89306bafc33e service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:09.530456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 661/1325] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:06:11.596468 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1aec5d-4c22-453d-98c7-61dd36a7653b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:11.597026 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1aec5d-4c22-453d-98c7-61dd36a7653b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:11.597026 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1aec5d-4c22-453d-98c7-61dd36a7653b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:11.598876 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 665/1326] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:06:11.604163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3fb0f-1c88-4a9a-be9a-6192a7d24ab5 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:11.604632 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3fb0f-1c88-4a9a-be9a-6192a7d24ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:11.604802 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3fb0f-1c88-4a9a-be9a-6192a7d24ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:11.674116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b7e3fb0f-1c88-4a9a-be9a-6192a7d24ab5 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:11.675370 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b7e3fb0f-1c88-4a9a-be9a-6192a7d24ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:11.776527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 662/1327] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:06:11.813132 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1aec5d-4c22-453d-98c7-61dd36a7653b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:11.814815 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5e1aec5d-4c22-453d-98c7-61dd36a7653b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:11.876877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:11.921999 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1aec5d-4c22-453d-98c7-61dd36a7653b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:11.998164 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2aa0af-cace-4353-8d89-dce10cf739b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:11.998791 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2aa0af-cace-4353-8d89-dce10cf739b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:11.999067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2aa0af-cace-4353-8d89-dce10cf739b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:11.999334 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2aa0af-cace-4353-8d89-dce10cf739b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:12.000936 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2c2aa0af-cace-4353-8d89-dce10cf739b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:12.112081 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:12.114554 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2aa0af-cace-4353-8d89-dce10cf739b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:12.116014 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2aa0af-cace-4353-8d89-dce10cf739b3 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:12.143484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 666/1328] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:06:14.240873 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944c142-faa5-4c76-9cdf-e11b2fe6859a None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:14.241407 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944c142-faa5-4c76-9cdf-e11b2fe6859a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:14.241407 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944c142-faa5-4c76-9cdf-e11b2fe6859a None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:14.242808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 663/1329] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:06:14.248073 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e2c92-2569-4927-bf48-1e12f171ca4b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:14.248797 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e2c92-2569-4927-bf48-1e12f171ca4b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:14.248797 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e2c92-2569-4927-bf48-1e12f171ca4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:14.316955 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-908e2c92-2569-4927-bf48-1e12f171ca4b None None] MFA Rules not processed for user `b98eacd3b6474daabcffaf3ef31aea2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:14.318157 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-908e2c92-2569-4927-bf48-1e12f171ca4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:14.420960 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 667/1330] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:06:14 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:14.458105 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944c142-faa5-4c76-9cdf-e11b2fe6859a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:14.459689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4944c142-faa5-4c76-9cdf-e11b2fe6859a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:14.490972 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:14.570530 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944c142-faa5-4c76-9cdf-e11b2fe6859a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:14.651172 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2dbbdf-bd82-46cd-a045-1d31e49a482f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:14.651835 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dbbdf-bd82-46cd-a045-1d31e49a482f service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:14.652086 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dbbdf-bd82-46cd-a045-1d31e49a482f service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:14.652354 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dbbdf-bd82-46cd-a045-1d31e49a482f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:14.654806 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1c2dbbdf-bd82-46cd-a045-1d31e49a482f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:14.731616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:14.766133 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2dbbdf-bd82-46cd-a045-1d31e49a482f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:14.767872 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2dbbdf-bd82-46cd-a045-1d31e49a482f service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:14.786037 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 664/1331] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:14 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:06:17.326300 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a8128-fae0-4e97-8169-54e9c73e8508 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:17.326859 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a8128-fae0-4e97-8169-54e9c73e8508 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:17.326859 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a8128-fae0-4e97-8169-54e9c73e8508 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:17.328189 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 668/1332] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:06:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:06:17.333446 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdad466-f0db-44cb-a974-2d9b8781cd61 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:17.333850 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdad466-f0db-44cb-a974-2d9b8781cd61 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:17.334019 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdad466-f0db-44cb-a974-2d9b8781cd61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:17.400918 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2fdad466-f0db-44cb-a974-2d9b8781cd61 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:17.402124 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2fdad466-f0db-44cb-a974-2d9b8781cd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:17.497127 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 665/1333] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:17.534187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a8128-fae0-4e97-8169-54e9c73e8508 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:17.535840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-423a8128-fae0-4e97-8169-54e9c73e8508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:17.606678 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:17.642302 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a8128-fae0-4e97-8169-54e9c73e8508 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:17.717277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef9420-2e5d-4038-972d-6546231669b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:17.717863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef9420-2e5d-4038-972d-6546231669b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:17.718140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef9420-2e5d-4038-972d-6546231669b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:17.718442 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef9420-2e5d-4038-972d-6546231669b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:17.720043 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-16ef9420-2e5d-4038-972d-6546231669b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:17.830250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef9420-2e5d-4038-972d-6546231669b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:17.831732 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef9420-2e5d-4038-972d-6546231669b3 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:17.841886 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:17.850159 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 669/1334] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:06:19.911958 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7514bc-c6a3-4cfb-a72c-b878b67c9a0a None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:19.912505 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7514bc-c6a3-4cfb-a72c-b878b67c9a0a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:19.912505 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7514bc-c6a3-4cfb-a72c-b878b67c9a0a None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:19.913768 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 666/1335] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:06:19.918907 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b782a-9c96-46be-a92e-c2ddd5c03ac7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:19.919381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b782a-9c96-46be-a92e-c2ddd5c03ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:19.919548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b782a-9c96-46be-a92e-c2ddd5c03ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:19.985308 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3b6b782a-9c96-46be-a92e-c2ddd5c03ac7 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:19.986537 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3b6b782a-9c96-46be-a92e-c2ddd5c03ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:20.081688 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 670/1336] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:20.123329 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7514bc-c6a3-4cfb-a72c-b878b67c9a0a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:20.124899 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-eb7514bc-c6a3-4cfb-a72c-b878b67c9a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:20.165633 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:20.228342 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7514bc-c6a3-4cfb-a72c-b878b67c9a0a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:20.300132 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8ff4f-0bdf-4df2-b130-71153e072647 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:20.300729 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8ff4f-0bdf-4df2-b130-71153e072647 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:20.301003 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8ff4f-0bdf-4df2-b130-71153e072647 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:20.301273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8ff4f-0bdf-4df2-b130-71153e072647 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:20.302906 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bfa8ff4f-0bdf-4df2-b130-71153e072647 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:20.384872 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:20.410203 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa8ff4f-0bdf-4df2-b130-71153e072647 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:20.411658 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa8ff4f-0bdf-4df2-b130-71153e072647 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:20.429200 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 667/1337] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:06:23.395860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ec202-2453-40c0-8cc9-c237e932eef0 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:23.396433 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ec202-2453-40c0-8cc9-c237e932eef0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:23.396433 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ec202-2453-40c0-8cc9-c237e932eef0 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:23.397745 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 671/1338] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:06:23.403033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f68be-989c-4426-8cac-6f3c64022a52 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:23.403504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f68be-989c-4426-8cac-6f3c64022a52 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:23.404128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f68be-989c-4426-8cac-6f3c64022a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:23.469473 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f00f68be-989c-4426-8cac-6f3c64022a52 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:23.470714 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f00f68be-989c-4426-8cac-6f3c64022a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:23.565773 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 668/1339] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:06:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:23.602784 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ec202-2453-40c0-8cc9-c237e932eef0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:23.604362 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-330ec202-2453-40c0-8cc9-c237e932eef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:23.680427 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:23.709370 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ec202-2453-40c0-8cc9-c237e932eef0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:23.782549 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1281ac-da03-4e4b-9c03-72240ccc6cff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:23.783150 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1281ac-da03-4e4b-9c03-72240ccc6cff service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:23.783424 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1281ac-da03-4e4b-9c03-72240ccc6cff service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:23.783690 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1281ac-da03-4e4b-9c03-72240ccc6cff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:23.785275 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cd1281ac-da03-4e4b-9c03-72240ccc6cff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:23.891874 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1281ac-da03-4e4b-9c03-72240ccc6cff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:23.893566 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1281ac-da03-4e4b-9c03-72240ccc6cff service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:23.911699 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 672/1340] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:06:28.396691 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836fcc9-26a4-47da-b551-c8bb42c49e6c None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:28.397478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836fcc9-26a4-47da-b551-c8bb42c49e6c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:28.397666 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836fcc9-26a4-47da-b551-c8bb42c49e6c None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:28.399252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 669/1341] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:06:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:06:28.405298 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8ca1d-8657-4632-97ff-d358e67f6376 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:28.405812 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8ca1d-8657-4632-97ff-d358e67f6376 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:28.406090 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8ca1d-8657-4632-97ff-d358e67f6376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:28.412386 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:28.476291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f2f8ca1d-8657-4632-97ff-d358e67f6376 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:28.477500 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f2f8ca1d-8657-4632-97ff-d358e67f6376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:28.576884 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 673/1342] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:06:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:28.616643 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836fcc9-26a4-47da-b551-c8bb42c49e6c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:28.618175 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c836fcc9-26a4-47da-b551-c8bb42c49e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:28.668487 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:28.726403 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836fcc9-26a4-47da-b551-c8bb42c49e6c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:28.802301 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f7d42-b2ee-4231-bd26-ce18048ba45d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:28.802972 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f7d42-b2ee-4231-bd26-ce18048ba45d service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:28.803245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f7d42-b2ee-4231-bd26-ce18048ba45d service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:28.803513 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f7d42-b2ee-4231-bd26-ce18048ba45d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:28.805171 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-eb4f7d42-b2ee-4231-bd26-ce18048ba45d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:28.905648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:28.927946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4f7d42-b2ee-4231-bd26-ce18048ba45d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:28.927946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4f7d42-b2ee-4231-bd26-ce18048ba45d service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:28.943268 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 670/1343] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:06:32.068234 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66878bf-64b2-4517-ace8-2d61d9c05e45 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:32.068796 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66878bf-64b2-4517-ace8-2d61d9c05e45 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:32.068796 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66878bf-64b2-4517-ace8-2d61d9c05e45 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:32.070054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 674/1344] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:06:32.075439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439dae8-8f43-40af-ac6d-4d160fec0e23 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:32.075862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439dae8-8f43-40af-ac6d-4d160fec0e23 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:32.075956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439dae8-8f43-40af-ac6d-4d160fec0e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:32.142946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d439dae8-8f43-40af-ac6d-4d160fec0e23 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:32.144153 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d439dae8-8f43-40af-ac6d-4d160fec0e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:32.243512 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 671/1345] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:06:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:32.282648 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66878bf-64b2-4517-ace8-2d61d9c05e45 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:32.284191 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e66878bf-64b2-4517-ace8-2d61d9c05e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:32.376143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:32.394166 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66878bf-64b2-4517-ace8-2d61d9c05e45 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:32.471726 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff88e37-0a30-44d9-a324-8df8af4baf4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:32.472327 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff88e37-0a30-44d9-a324-8df8af4baf4e service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:32.472617 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff88e37-0a30-44d9-a324-8df8af4baf4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:32.472879 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff88e37-0a30-44d9-a324-8df8af4baf4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:32.474519 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8ff88e37-0a30-44d9-a324-8df8af4baf4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:32.597251 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff88e37-0a30-44d9-a324-8df8af4baf4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:32.598281 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff88e37-0a30-44d9-a324-8df8af4baf4e service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:32.619960 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 675/1346] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:06:36.920158 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e521c30-ef69-4873-9969-767bc60cd28e None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:36.920735 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e521c30-ef69-4873-9969-767bc60cd28e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:36.920735 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e521c30-ef69-4873-9969-767bc60cd28e None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:36.922062 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 672/1347] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:06:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:06:36.927638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d490a89-4bac-4950-8e4e-0b76889090ba None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:36.928110 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d490a89-4bac-4950-8e4e-0b76889090ba None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:36.928225 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d490a89-4bac-4950-8e4e-0b76889090ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:36.944647 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:37.001086 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0d490a89-4bac-4950-8e4e-0b76889090ba None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:37.002323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0d490a89-4bac-4950-8e4e-0b76889090ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:37.103641 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 676/1348] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:37.143345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e521c30-ef69-4873-9969-767bc60cd28e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:37.144978 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4e521c30-ef69-4873-9969-767bc60cd28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:37.205627 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:37.256769 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e521c30-ef69-4873-9969-767bc60cd28e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:37.332830 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e8480-e582-44a5-bb9e-c85e0322af58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:37.333428 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e8480-e582-44a5-bb9e-c85e0322af58 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:37.333728 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e8480-e582-44a5-bb9e-c85e0322af58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:37.333967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e8480-e582-44a5-bb9e-c85e0322af58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:37.335620 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7a3e8480-e582-44a5-bb9e-c85e0322af58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:37.449757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:37.453303 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3e8480-e582-44a5-bb9e-c85e0322af58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:37.454767 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3e8480-e582-44a5-bb9e-c85e0322af58 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:37.473327 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 673/1349] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:06:42.519185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7ed79-39a9-4a1c-b992-7c6b0a990de0 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:42.519626 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7ed79-39a9-4a1c-b992-7c6b0a990de0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:42.519718 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7ed79-39a9-4a1c-b992-7c6b0a990de0 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:42.521488 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 677/1350] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 10 17:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:06:42.527735 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64e0f6-d210-4868-958e-570412c29017 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:42.528232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64e0f6-d210-4868-958e-570412c29017 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:42.528323 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64e0f6-d210-4868-958e-570412c29017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:42.615461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ca64e0f6-d210-4868-958e-570412c29017 None None] MFA Rules not processed for user `8cd39d9a583a4d73a5654017be106019`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:42.616993 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ca64e0f6-d210-4868-958e-570412c29017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:42.797602 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 674/1351] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 10 17:06:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:46.922945 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f25be3-27b2-43d9-b01f-7a838a577605 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:46.923460 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f25be3-27b2-43d9-b01f-7a838a577605 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:46.923460 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f25be3-27b2-43d9-b01f-7a838a577605 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:46.924960 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 678/1352] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 10 17:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:06:46.929793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a1386-cd9c-46d4-8cc5-045072a43b05 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:46.930666 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a1386-cd9c-46d4-8cc5-045072a43b05 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:46.931159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a1386-cd9c-46d4-8cc5-045072a43b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:46.992267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:47.018565 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b18a1386-cd9c-46d4-8cc5-045072a43b05 None None] MFA Rules not processed for user `8cd39d9a583a4d73a5654017be106019`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:47.020331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b18a1386-cd9c-46d4-8cc5-045072a43b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:47.101602 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=59 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:47.137431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 675/1353] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 10 17:06:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:48.666443 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d522bc8-7bd4-4568-956c-0c21da3fdee0 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:48.667001 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d522bc8-7bd4-4568-956c-0c21da3fdee0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:48.667101 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d522bc8-7bd4-4568-956c-0c21da3fdee0 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:48.668560 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 679/1354] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 10 17:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:06:48.674239 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86e268-d745-43c0-932b-bbf2f1268b59 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:48.674709 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86e268-d745-43c0-932b-bbf2f1268b59 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:48.674808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86e268-d745-43c0-932b-bbf2f1268b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:48.752655 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6f86e268-d745-43c0-932b-bbf2f1268b59 None None] MFA Rules not processed for user `8cd39d9a583a4d73a5654017be106019`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:48.753869 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6f86e268-d745-43c0-932b-bbf2f1268b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:48.797151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:48.867692 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 676/1355] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 10 17:06:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:52.392658 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80059a4-b119-4f0b-be36-55d3efacc237 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:52.393153 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80059a4-b119-4f0b-be36-55d3efacc237 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:52.393252 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80059a4-b119-4f0b-be36-55d3efacc237 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:52.394557 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 680/1356] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:06:52.399834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac270a-9f3a-4acc-851f-3cff4614add0 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:52.400282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac270a-9f3a-4acc-851f-3cff4614add0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:52.400500 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac270a-9f3a-4acc-851f-3cff4614add0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:52.493260 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-43ac270a-9f3a-4acc-851f-3cff4614add0 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:52.494631 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-43ac270a-9f3a-4acc-851f-3cff4614add0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:52.619947 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 677/1357] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:52.675143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2ae93-3533-4a62-89ee-de7ddac54b05 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:52.675760 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2ae93-3533-4a62-89ee-de7ddac54b05 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:52.676659 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2ae93-3533-4a62-89ee-de7ddac54b05 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:52.678283 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 681/1358] 10.222.0.45 () {54 vars in 962 bytes} [Wed Aug 10 17:06:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:06:52.694011 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521f9eb-b838-4aa2-9a9d-b5aaea7dd082 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:52.694011 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521f9eb-b838-4aa2-9a9d-b5aaea7dd082 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:52.694011 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521f9eb-b838-4aa2-9a9d-b5aaea7dd082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:52.706629 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:52.771503 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8521f9eb-b838-4aa2-9a9d-b5aaea7dd082 None None] MFA Rules not processed for user `615cc36e21cc493b823e6cee695ac308`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:52.772774 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8521f9eb-b838-4aa2-9a9d-b5aaea7dd082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:52.909278 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 678/1359] 10.222.0.45 () {58 vars in 1059 bytes} [Wed Aug 10 17:06:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:52.917557 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2ae93-3533-4a62-89ee-de7ddac54b05 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:52.919887 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-63e2ae93-3533-4a62-89ee-de7ddac54b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:53.079400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2ae93-3533-4a62-89ee-de7ddac54b05 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:53.174280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:53.215357 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6099280-9e05-45f5-b194-d8f5cf1954d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:53.216910 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6099280-9e05-45f5-b194-d8f5cf1954d7 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:53.217661 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6099280-9e05-45f5-b194-d8f5cf1954d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:53.217947 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6099280-9e05-45f5-b194-d8f5cf1954d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:53.220310 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a6099280-9e05-45f5-b194-d8f5cf1954d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:53.322267 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6099280-9e05-45f5-b194-d8f5cf1954d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:53.323957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6099280-9e05-45f5-b194-d8f5cf1954d7 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:53.344894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 682/1360] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:06:55.651999 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360c136-2034-4ac9-8c6f-0a9271330c6f None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:55.652621 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360c136-2034-4ac9-8c6f-0a9271330c6f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:55.652621 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360c136-2034-4ac9-8c6f-0a9271330c6f None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:55.654183 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 679/1361] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:06:55.659449 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00652439-62ac-4652-847c-a73a8b3c9a6c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:55.659953 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00652439-62ac-4652-847c-a73a8b3c9a6c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:55.660034 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00652439-62ac-4652-847c-a73a8b3c9a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:55.742201 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-00652439-62ac-4652-847c-a73a8b3c9a6c None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:55.743493 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-00652439-62ac-4652-847c-a73a8b3c9a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:55.792716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:55.866084 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 683/1362] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:55.924171 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0360c136-2034-4ac9-8c6f-0a9271330c6f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:55.925698 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0360c136-2034-4ac9-8c6f-0a9271330c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:56.021886 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:56.139876 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0360c136-2034-4ac9-8c6f-0a9271330c6f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:56.310023 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fccafb1-9ba8-4a02-8e01-80b06d27a63d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:56.310708 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccafb1-9ba8-4a02-8e01-80b06d27a63d service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:56.310999 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccafb1-9ba8-4a02-8e01-80b06d27a63d service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:56.311290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccafb1-9ba8-4a02-8e01-80b06d27a63d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:56.312915 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6fccafb1-9ba8-4a02-8e01-80b06d27a63d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:56.340785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:56.431860 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fccafb1-9ba8-4a02-8e01-80b06d27a63d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:56.433855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fccafb1-9ba8-4a02-8e01-80b06d27a63d service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:56.458770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 680/1363] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:06:59.024639 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b8ae1-e669-41cb-93c8-8927ba7d8710 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:59.025490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b8ae1-e669-41cb-93c8-8927ba7d8710 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:59.025760 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b8ae1-e669-41cb-93c8-8927ba7d8710 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:59.027146 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 684/1364] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:06:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:06:59.032970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172fb103-388d-44ca-ac7a-9f7da899e28f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:59.033506 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172fb103-388d-44ca-ac7a-9f7da899e28f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:59.033506 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172fb103-388d-44ca-ac7a-9f7da899e28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:59.108342 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-172fb103-388d-44ca-ac7a-9f7da899e28f None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:59.123268 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-172fb103-388d-44ca-ac7a-9f7da899e28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:59.216211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:59.219360 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 681/1365] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:59.272131 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4e33c-0a34-4d6f-887f-e1add20e47c9 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:59.272510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4e33c-0a34-4d6f-887f-e1add20e47c9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:59.272855 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4e33c-0a34-4d6f-887f-e1add20e47c9 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:59.274296 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 685/1366] 10.222.0.45 () {54 vars in 962 bytes} [Wed Aug 10 17:06:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 17:06:59.283635 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b6586-a732-408c-be16-480e3d831a1b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:59.283635 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b6586-a732-408c-be16-480e3d831a1b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:59.283635 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b6586-a732-408c-be16-480e3d831a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:59.374340 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-de4b6586-a732-408c-be16-480e3d831a1b None None] MFA Rules not processed for user `615cc36e21cc493b823e6cee695ac308`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:06:59.375736 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-de4b6586-a732-408c-be16-480e3d831a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:59.510234 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 682/1367] 10.222.0.45 () {58 vars in 1059 bytes} [Wed Aug 10 17:06:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:06:59.519093 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4e33c-0a34-4d6f-887f-e1add20e47c9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:06:59.520647 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f9d4e33c-0a34-4d6f-887f-e1add20e47c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:59.692919 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4e33c-0a34-4d6f-887f-e1add20e47c9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:06:59.717671 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:06:59.818957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd1428-1569-461d-b050-17f90c3223b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:06:59.819565 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd1428-1569-461d-b050-17f90c3223b0 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:06:59.819839 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd1428-1569-461d-b050-17f90c3223b0 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:06:59.820104 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd1428-1569-461d-b050-17f90c3223b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:06:59.821740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4ebd1428-1569-461d-b050-17f90c3223b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:06:59.923696 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd1428-1569-461d-b050-17f90c3223b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:06:59.925226 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd1428-1569-461d-b050-17f90c3223b0 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:06:59.945624 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 686/1368] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:06:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:00.845319 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e5819-b27d-432e-ac44-f3f26fccacb8 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:00.845939 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e5819-b27d-432e-ac44-f3f26fccacb8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:00.845939 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e5819-b27d-432e-ac44-f3f26fccacb8 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:00.847433 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 683/1369] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:07:00.852905 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45404e-cfc7-4bf0-9de3-d8df0e869b45 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:00.853393 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45404e-cfc7-4bf0-9de3-d8df0e869b45 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:00.853508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45404e-cfc7-4bf0-9de3-d8df0e869b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:00.889198 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:00.928252 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2e45404e-cfc7-4bf0-9de3-d8df0e869b45 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:00.929488 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2e45404e-cfc7-4bf0-9de3-d8df0e869b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:01.046585 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 687/1370] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:07:01.406608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da929af4-1fde-4580-b974-83b91c4d00ee None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:01.406608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da929af4-1fde-4580-b974-83b91c4d00ee None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:01.406608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da929af4-1fde-4580-b974-83b91c4d00ee None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:01.407618 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 684/1371] 10.222.0.45 () {54 vars in 964 bytes} [Wed Aug 10 17:07:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:07:01.412027 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53005def-6721-454e-8ac2-88642a855321 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:01.412315 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53005def-6721-454e-8ac2-88642a855321 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:01.412589 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53005def-6721-454e-8ac2-88642a855321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:01.499784 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-53005def-6721-454e-8ac2-88642a855321 None None] MFA Rules not processed for user `56d5c4afb9844fbbbe699f958deae6e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:01.501109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-53005def-6721-454e-8ac2-88642a855321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:01.555263 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:01.588071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 688/1372] 10.222.0.45 () {58 vars in 1061 bytes} [Wed Aug 10 17:07:01 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:01.597467 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da929af4-1fde-4580-b974-83b91c4d00ee None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:01.599150 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-da929af4-1fde-4580-b974-83b91c4d00ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:01.668678 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:01.721471 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da929af4-1fde-4580-b974-83b91c4d00ee None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:01.802921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42234ef3-5ea0-40f5-824f-5c71f3c633fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:01.803667 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42234ef3-5ea0-40f5-824f-5c71f3c633fd service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:01.804014 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42234ef3-5ea0-40f5-824f-5c71f3c633fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:01.804362 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42234ef3-5ea0-40f5-824f-5c71f3c633fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:01.806167 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-42234ef3-5ea0-40f5-824f-5c71f3c633fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:01.926172 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:01.942241 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42234ef3-5ea0-40f5-824f-5c71f3c633fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:01.949645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42234ef3-5ea0-40f5-824f-5c71f3c633fd service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:01.963571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 685/1373] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:07:02.253568 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000778d-52dd-4527-9188-a5f39f68cde3 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:02.254022 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000778d-52dd-4527-9188-a5f39f68cde3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:02.254257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000778d-52dd-4527-9188-a5f39f68cde3 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:02.259700 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 689/1374] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:07:02.260831 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693e6ca-1ead-4836-9ccf-82d830def628 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:02.261603 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693e6ca-1ead-4836-9ccf-82d830def628 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:02.262021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693e6ca-1ead-4836-9ccf-82d830def628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:02.349419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b693e6ca-1ead-4836-9ccf-82d830def628 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:02.350705 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b693e6ca-1ead-4836-9ccf-82d830def628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:02.468820 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 686/1375] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:02.514816 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000778d-52dd-4527-9188-a5f39f68cde3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:02.516492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4000778d-52dd-4527-9188-a5f39f68cde3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:02.672173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000778d-52dd-4527-9188-a5f39f68cde3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:02.724978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:02.790261 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08895cd2-14c8-4ce9-b6a2-8fdb6914f5e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:02.790896 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08895cd2-14c8-4ce9-b6a2-8fdb6914f5e1 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:02.791229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08895cd2-14c8-4ce9-b6a2-8fdb6914f5e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:02.791501 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08895cd2-14c8-4ce9-b6a2-8fdb6914f5e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:02.793190 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-08895cd2-14c8-4ce9-b6a2-8fdb6914f5e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:02.888020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08895cd2-14c8-4ce9-b6a2-8fdb6914f5e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:02.889613 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08895cd2-14c8-4ce9-b6a2-8fdb6914f5e1 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:02.909580 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 690/1376] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:05.264377 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43f1e0-724c-4949-a739-bdbdbce35a60 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:05.264943 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43f1e0-724c-4949-a739-bdbdbce35a60 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:05.264943 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43f1e0-724c-4949-a739-bdbdbce35a60 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:05.270402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 687/1377] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:07:05.276452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c693de-be82-4d9f-9a7d-497ac02fd46b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:05.276452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c693de-be82-4d9f-9a7d-497ac02fd46b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:05.276452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c693de-be82-4d9f-9a7d-497ac02fd46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:05.352657 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-71c693de-be82-4d9f-9a7d-497ac02fd46b None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:05.353878 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-71c693de-be82-4d9f-9a7d-497ac02fd46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:05.361116 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:05.475855 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 691/1378] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:05.521845 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43f1e0-724c-4949-a739-bdbdbce35a60 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:05.523332 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3f43f1e0-724c-4949-a739-bdbdbce35a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:05.586278 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:05.676248 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43f1e0-724c-4949-a739-bdbdbce35a60 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:05.845441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce2607-13ae-4b17-8bcc-1de28f276f91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:05.845946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce2607-13ae-4b17-8bcc-1de28f276f91 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:05.846220 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce2607-13ae-4b17-8bcc-1de28f276f91 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:05.846540 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce2607-13ae-4b17-8bcc-1de28f276f91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:05.848220 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-74ce2607-13ae-4b17-8bcc-1de28f276f91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:05.876864 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:05.973054 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce2607-13ae-4b17-8bcc-1de28f276f91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:05.975031 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce2607-13ae-4b17-8bcc-1de28f276f91 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:05.995930 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 688/1379] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:08.208044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa1350-cee4-459e-b232-0414987d1824 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:08.215921 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa1350-cee4-459e-b232-0414987d1824 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:08.215921 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa1350-cee4-459e-b232-0414987d1824 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:08.215921 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 692/1380] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:07:08.216217 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b4d78-58d6-4197-8a49-aae0437b8e67 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:08.216217 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b4d78-58d6-4197-8a49-aae0437b8e67 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:08.216701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b4d78-58d6-4197-8a49-aae0437b8e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:08.294052 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-909b4d78-58d6-4197-8a49-aae0437b8e67 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:08.295485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-909b4d78-58d6-4197-8a49-aae0437b8e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:08.401312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:08.409253 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 689/1381] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:08.456291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa1350-cee4-459e-b232-0414987d1824 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:08.457851 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-64fa1350-cee4-459e-b232-0414987d1824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:08.613637 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:08.625905 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa1350-cee4-459e-b232-0414987d1824 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:08.763146 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ea560-46ef-4010-b0a3-8a54a7f0e275 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:08.763684 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ea560-46ef-4010-b0a3-8a54a7f0e275 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:08.763961 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ea560-46ef-4010-b0a3-8a54a7f0e275 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:08.764231 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ea560-46ef-4010-b0a3-8a54a7f0e275 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:08.766034 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8b0ea560-46ef-4010-b0a3-8a54a7f0e275 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:08.865658 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ea560-46ef-4010-b0a3-8a54a7f0e275 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:08.867216 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ea560-46ef-4010-b0a3-8a54a7f0e275 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:08.881213 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:08.890866 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 693/1382] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:07:11.105609 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2eed85-a272-4a08-a403-958d8951f4da None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:11.106901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2eed85-a272-4a08-a403-958d8951f4da None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:11.106901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2eed85-a272-4a08-a403-958d8951f4da None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:11.108418 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 690/1383] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:07:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 17:07:11.114525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f6399-2059-4217-8096-c6e7984adcee None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:11.115230 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f6399-2059-4217-8096-c6e7984adcee None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:11.115230 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f6399-2059-4217-8096-c6e7984adcee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:11.200794 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a00f6399-2059-4217-8096-c6e7984adcee None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:11.202600 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a00f6399-2059-4217-8096-c6e7984adcee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:11.331756 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 694/1384] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:11.379185 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2eed85-a272-4a08-a403-958d8951f4da None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:11.381478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5e2eed85-a272-4a08-a403-958d8951f4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:11.549801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2eed85-a272-4a08-a403-958d8951f4da None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:11.656512 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:11.733077 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7e2bb-1649-46c1-89fd-b1d350397032 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:11.733686 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7e2bb-1649-46c1-89fd-b1d350397032 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:11.733984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7e2bb-1649-46c1-89fd-b1d350397032 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:11.734251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7e2bb-1649-46c1-89fd-b1d350397032 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:11.735966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f6e7e2bb-1649-46c1-89fd-b1d350397032 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:11.851135 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7e2bb-1649-46c1-89fd-b1d350397032 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:11.854591 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7e2bb-1649-46c1-89fd-b1d350397032 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:11.894146 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 691/1385] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:13.953324 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29cc88-feac-462c-84a9-b824c13a7139 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:13.953864 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29cc88-feac-462c-84a9-b824c13a7139 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:13.953864 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29cc88-feac-462c-84a9-b824c13a7139 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:13.955605 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 695/1386] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:07:13.961088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676ed5c-1b26-4e94-be30-813cc69ddcd8 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:13.961564 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676ed5c-1b26-4e94-be30-813cc69ddcd8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:13.961665 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676ed5c-1b26-4e94-be30-813cc69ddcd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:14.031567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c676ed5c-1b26-4e94-be30-813cc69ddcd8 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:14.032839 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c676ed5c-1b26-4e94-be30-813cc69ddcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:14.068412 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:14.144137 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 692/1387] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:07:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:07:14.196010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de29cc88-feac-462c-84a9-b824c13a7139 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:14.197570 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-de29cc88-feac-462c-84a9-b824c13a7139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:14.283025 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:14.366913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de29cc88-feac-462c-84a9-b824c13a7139 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:14.490467 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47025dbe-6e8e-4056-9d8a-46c5c6231c64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:14.491316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47025dbe-6e8e-4056-9d8a-46c5c6231c64 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:14.491689 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47025dbe-6e8e-4056-9d8a-46c5c6231c64 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:14.492077 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47025dbe-6e8e-4056-9d8a-46c5c6231c64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:14.494391 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-47025dbe-6e8e-4056-9d8a-46c5c6231c64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:14.539799 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:14.594847 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47025dbe-6e8e-4056-9d8a-46c5c6231c64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:14.596450 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47025dbe-6e8e-4056-9d8a-46c5c6231c64 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:14.618390 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 696/1388] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:07:16.842325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84b856-992e-43ec-8468-6c07f8679000 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:16.842325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84b856-992e-43ec-8468-6c07f8679000 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:16.843052 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84b856-992e-43ec-8468-6c07f8679000 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:16.843944 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 693/1389] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:07:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:07:16.854941 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8de64-d0e2-46a6-83ea-c420cfe69613 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:16.855435 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8de64-d0e2-46a6-83ea-c420cfe69613 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:16.855572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8de64-d0e2-46a6-83ea-c420cfe69613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:16.925894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-29d8de64-d0e2-46a6-83ea-c420cfe69613 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:16.927076 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-29d8de64-d0e2-46a6-83ea-c420cfe69613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:17.025204 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 697/1390] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:17.070457 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84b856-992e-43ec-8468-6c07f8679000 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:17.071984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ea84b856-992e-43ec-8468-6c07f8679000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:17.227707 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84b856-992e-43ec-8468-6c07f8679000 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:17.245834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:17.345072 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a9dd8-1ec6-4693-9349-15f03de77e59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:17.345676 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a9dd8-1ec6-4693-9349-15f03de77e59 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:17.345950 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a9dd8-1ec6-4693-9349-15f03de77e59 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:17.346219 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a9dd8-1ec6-4693-9349-15f03de77e59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:17.347859 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-570a9dd8-1ec6-4693-9349-15f03de77e59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:17.443069 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a9dd8-1ec6-4693-9349-15f03de77e59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:17.444682 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a9dd8-1ec6-4693-9349-15f03de77e59 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:17.463624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 694/1391] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:18.016284 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96196793-e512-4c06-b21c-91c8be715cce None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:18.017115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96196793-e512-4c06-b21c-91c8be715cce None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:18.017115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96196793-e512-4c06-b21c-91c8be715cce None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:18.018454 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 698/1392] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:07:18.023848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be4277-a150-4a33-8474-06cf039c2178 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:18.024307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be4277-a150-4a33-8474-06cf039c2178 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:18.024403 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be4277-a150-4a33-8474-06cf039c2178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:18.051975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:18.094144 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-99be4277-a150-4a33-8474-06cf039c2178 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:18.095400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-99be4277-a150-4a33-8474-06cf039c2178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:18.196340 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 695/1393] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:18.255936 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40c0f2-055d-482a-9115-094fb87b810a None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:18.256212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40c0f2-055d-482a-9115-094fb87b810a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:18.256480 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40c0f2-055d-482a-9115-094fb87b810a None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:18.257809 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 699/1394] 10.222.0.45 () {54 vars in 967 bytes} [Wed Aug 10 17:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:07:18.263576 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f1a45-6e8c-45ac-be03-c2c887503295 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:18.264264 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f1a45-6e8c-45ac-be03-c2c887503295 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:18.264572 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f1a45-6e8c-45ac-be03-c2c887503295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:18.344046 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a79f1a45-6e8c-45ac-be03-c2c887503295 None None] MFA Rules not processed for user `10f4f54890624da29ff217081445b6bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:18.345266 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a79f1a45-6e8c-45ac-be03-c2c887503295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:18.389463 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:18.427557 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 696/1395] 10.222.0.45 () {58 vars in 1064 bytes} [Wed Aug 10 17:07:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:18.435888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40c0f2-055d-482a-9115-094fb87b810a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:18.437528 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9a40c0f2-055d-482a-9115-094fb87b810a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:18.452256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:18.547683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40c0f2-055d-482a-9115-094fb87b810a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:18.634422 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b5458-8f98-4faa-81c6-30a40a7adaf5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10f4f54890624da29ff217081445b6bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:18.635024 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b5458-8f98-4faa-81c6-30a40a7adaf5 service glance] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:18.635296 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b5458-8f98-4faa-81c6-30a40a7adaf5 service glance] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:18.635560 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b5458-8f98-4faa-81c6-30a40a7adaf5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:18.638478 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c93b5458-8f98-4faa-81c6-30a40a7adaf5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:18.709980 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:18.772181 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b5458-8f98-4faa-81c6-30a40a7adaf5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:18.774911 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b5458-8f98-4faa-81c6-30a40a7adaf5 service glance] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:18.804986 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 700/1396] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:19.668158 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734acf8-8f18-4927-a538-2a674cd8aa69 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:19.668744 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734acf8-8f18-4927-a538-2a674cd8aa69 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:19.668845 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734acf8-8f18-4927-a538-2a674cd8aa69 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:19.670170 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 697/1397] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:07:19.675616 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352612e-27a8-49ce-a4d5-c032b273afcc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:19.677762 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352612e-27a8-49ce-a4d5-c032b273afcc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:19.677762 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352612e-27a8-49ce-a4d5-c032b273afcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:19.709252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb126c1f-bbfa-453a-88c5-b667117df543 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:19.710781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb126c1f-bbfa-453a-88c5-b667117df543 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:19.710781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb126c1f-bbfa-453a-88c5-b667117df543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:19.755379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9352612e-27a8-49ce-a4d5-c032b273afcc None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:19.756865 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9352612e-27a8-49ce-a4d5-c032b273afcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:19.779980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-bb126c1f-bbfa-453a-88c5-b667117df543 None None] MFA Rules not processed for user `4f9e1767ec90435f80248a2930081265`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:19.781202 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bb126c1f-bbfa-453a-88c5-b667117df543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:19.866577 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 701/1398] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:19.873406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 698/1399] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 10 17:07:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:19.912928 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352612e-27a8-49ce-a4d5-c032b273afcc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:19.914143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9352612e-27a8-49ce-a4d5-c032b273afcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:19.919041 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6f86c-3b14-44a6-aaa1-825719d211ea None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:19.919379 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6f86c-3b14-44a6-aaa1-825719d211ea None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:19.919651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6f86c-3b14-44a6-aaa1-825719d211ea None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:19.921028 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 699/1400] 10.222.0.45 () {54 vars in 943 bytes} [Wed Aug 10 17:07:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 17:07:19.927580 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a175a-b76f-46d3-bc0e-b7ee0bb56108 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:19.927859 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a175a-b76f-46d3-bc0e-b7ee0bb56108 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:19.928127 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a175a-b76f-46d3-bc0e-b7ee0bb56108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:19.942957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:20.001503 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:20.008946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-501a175a-b76f-46d3-bc0e-b7ee0bb56108 None None] MFA Rules not processed for user `4986821ef9f84b99a835e387635aa9a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:20.010162 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-501a175a-b76f-46d3-bc0e-b7ee0bb56108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:20.034974 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352612e-27a8-49ce-a4d5-c032b273afcc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:20.125608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 700/1401] 10.222.0.45 () {58 vars in 1040 bytes} [Wed Aug 10 17:07:19 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:20.134760 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a175a-b76f-46d3-bc0e-b7ee0bb56108 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:20.145656 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a175a-b76f-46d3-bc0e-b7ee0bb56108 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:20.160583 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56822cba-19e8-4f0a-aa52-a461435fc73d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:20.161192 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56822cba-19e8-4f0a-aa52-a461435fc73d service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:20.161467 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56822cba-19e8-4f0a-aa52-a461435fc73d service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:20.161735 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56822cba-19e8-4f0a-aa52-a461435fc73d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:20.163383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-56822cba-19e8-4f0a-aa52-a461435fc73d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:20.181295 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:20.255561 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56822cba-19e8-4f0a-aa52-a461435fc73d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:20.256254 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8def12d-93d0-4bd5-b6c6-254bd695609a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4986821ef9f84b99a835e387635aa9a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:20.256863 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8def12d-93d0-4bd5-b6c6-254bd695609a service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:20.257137 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8def12d-93d0-4bd5-b6c6-254bd695609a service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:20.257247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56822cba-19e8-4f0a-aa52-a461435fc73d service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:20.257403 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8def12d-93d0-4bd5-b6c6-254bd695609a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:20.259107 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f8def12d-93d0-4bd5-b6c6-254bd695609a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:20.262690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:20.277813 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 702/1402] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:20.346409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8def12d-93d0-4bd5-b6c6-254bd695609a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:20.348074 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8def12d-93d0-4bd5-b6c6-254bd695609a service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:20.356363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 701/1403] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 10 17:07:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 17:07:22.528707 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c7d42-f969-41c4-90a5-4078b950552d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:22.529323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c7d42-f969-41c4-90a5-4078b950552d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:22.530029 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c7d42-f969-41c4-90a5-4078b950552d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:22.532447 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 703/1404] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:07:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:07:22.537862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e0336-881f-4849-993a-b1daae72de09 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:22.538434 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e0336-881f-4849-993a-b1daae72de09 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:22.538533 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e0336-881f-4849-993a-b1daae72de09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:22.610849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-476e0336-881f-4849-993a-b1daae72de09 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:22.615857 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-476e0336-881f-4849-993a-b1daae72de09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:22.720372 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:22.734810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 702/1405] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:22.775188 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c7d42-f969-41c4-90a5-4078b950552d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:22.788494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0e7c7d42-f969-41c4-90a5-4078b950552d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:22.943230 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:22.946864 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c7d42-f969-41c4-90a5-4078b950552d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:23.082562 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdb483-acbc-4316-91b9-1400aac03510 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:23.083181 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdb483-acbc-4316-91b9-1400aac03510 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:23.083505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdb483-acbc-4316-91b9-1400aac03510 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:23.083779 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdb483-acbc-4316-91b9-1400aac03510 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:23.085410 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b4fdb483-acbc-4316-91b9-1400aac03510 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:23.222070 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdb483-acbc-4316-91b9-1400aac03510 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:23.225099 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdb483-acbc-4316-91b9-1400aac03510 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:23.259468 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 704/1406] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:25.471375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed6a96-2efc-4305-a9f7-702bbf14957c None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:25.471966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed6a96-2efc-4305-a9f7-702bbf14957c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:25.471966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed6a96-2efc-4305-a9f7-702bbf14957c None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:25.473330 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 703/1407] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:07:25.478918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b9cad-9c79-4bdc-bc69-739b2e8ecd44 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:25.479451 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b9cad-9c79-4bdc-bc69-739b2e8ecd44 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:25.479451 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b9cad-9c79-4bdc-bc69-739b2e8ecd44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:25.484865 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:25.564482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-004b9cad-9c79-4bdc-bc69-739b2e8ecd44 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:25.565722 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-004b9cad-9c79-4bdc-bc69-739b2e8ecd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:25.694551 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 705/1408] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:25.740074 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed6a96-2efc-4305-a9f7-702bbf14957c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:25.741648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-42ed6a96-2efc-4305-a9f7-702bbf14957c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:25.928032 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed6a96-2efc-4305-a9f7-702bbf14957c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:26.021468 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:26.071695 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e3b1f-e329-49f8-ab4f-d8371343d7ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:26.072467 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e3b1f-e329-49f8-ab4f-d8371343d7ea service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:26.072790 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e3b1f-e329-49f8-ab4f-d8371343d7ea service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:26.073071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e3b1f-e329-49f8-ab4f-d8371343d7ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:26.074944 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ba4e3b1f-e329-49f8-ab4f-d8371343d7ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:26.197131 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e3b1f-e329-49f8-ab4f-d8371343d7ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:26.198906 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e3b1f-e329-49f8-ab4f-d8371343d7ea service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:26.229793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 704/1409] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:35.691855 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=64 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:36.225518 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=64 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:44.007783 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88db85-1f54-4462-a2bf-eea26fe7a53b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:44.008314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88db85-1f54-4462-a2bf-eea26fe7a53b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:44.008314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88db85-1f54-4462-a2bf-eea26fe7a53b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:44.009879 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 706/1410] 10.222.0.45 () {54 vars in 918 bytes} [Wed Aug 10 17:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:07:44.015125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159998d-49bb-48a8-8ca3-3976d4244644 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:44.015580 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159998d-49bb-48a8-8ca3-3976d4244644 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:44.015691 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159998d-49bb-48a8-8ca3-3976d4244644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:44.090809 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a159998d-49bb-48a8-8ca3-3976d4244644 None None] MFA Rules not processed for user `8cd39d9a583a4d73a5654017be106019`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:44.092025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a159998d-49bb-48a8-8ca3-3976d4244644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:44.197266 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 705/1411] 10.222.0.45 () {58 vars in 1015 bytes} [Wed Aug 10 17:07:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:44.212235 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ae474-4eeb-4c6c-bc14-ccea980edf5e None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:44.212529 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ae474-4eeb-4c6c-bc14-ccea980edf5e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:44.212782 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ae474-4eeb-4c6c-bc14-ccea980edf5e None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:44.214114 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 707/1412] 10.222.0.45 () {54 vars in 918 bytes} [Wed Aug 10 17:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:07:44.219404 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43334c-ae43-423a-b68c-50bb5f281619 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:44.219687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43334c-ae43-423a-b68c-50bb5f281619 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:44.219961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43334c-ae43-423a-b68c-50bb5f281619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:44.297558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:44.300789 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-db43334c-ae43-423a-b68c-50bb5f281619 None None] MFA Rules not processed for user `615cc36e21cc493b823e6cee695ac308`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:44.302058 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-db43334c-ae43-423a-b68c-50bb5f281619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:44.430971 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 706/1413] 10.222.0.45 () {58 vars in 1015 bytes} [Wed Aug 10 17:07:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:46.949445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae06d4-4c9f-4c17-a847-fb56d7333711 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:46.950016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae06d4-4c9f-4c17-a847-fb56d7333711 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:46.950173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae06d4-4c9f-4c17-a847-fb56d7333711 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:46.951553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 17:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 17:07:46.956968 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ec071-02b7-4afb-8b65-58c6a4bc6fba None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:46.957415 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ec071-02b7-4afb-8b65-58c6a4bc6fba None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:46.957543 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ec071-02b7-4afb-8b65-58c6a4bc6fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:47.150906 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-bb4ec071-02b7-4afb-8b65-58c6a4bc6fba None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:47.152132 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bb4ec071-02b7-4afb-8b65-58c6a4bc6fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:47.209245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:47.251565 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 17:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:47.294064 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae06d4-4c9f-4c17-a847-fb56d7333711 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:47.295697 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ffae06d4-4c9f-4c17-a847-fb56d7333711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:47.441891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae06d4-4c9f-4c17-a847-fb56d7333711 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:47.517942 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:47.557519 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfff3490-bd87-4a09-9785-052f6449db9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:47.558130 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff3490-bd87-4a09-9785-052f6449db9c service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:47.559050 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff3490-bd87-4a09-9785-052f6449db9c service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:47.559404 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff3490-bd87-4a09-9785-052f6449db9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:47.561014 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cfff3490-bd87-4a09-9785-052f6449db9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:47.653190 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfff3490-bd87-4a09-9785-052f6449db9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:47.654808 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfff3490-bd87-4a09-9785-052f6449db9c service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:47.673147 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 709/1416] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:48.584749 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d33e8-81cd-4821-aa2c-6f87f14cc7bf None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:48.585346 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d33e8-81cd-4821-aa2c-6f87f14cc7bf None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:48.585346 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d33e8-81cd-4821-aa2c-6f87f14cc7bf None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:48.586673 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 17:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:07:48.592348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504e818-fe9c-449a-9a96-35004d7cda82 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:48.592801 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504e818-fe9c-449a-9a96-35004d7cda82 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:48.592923 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504e818-fe9c-449a-9a96-35004d7cda82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:48.671343 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4504e818-fe9c-449a-9a96-35004d7cda82 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:48.672555 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4504e818-fe9c-449a-9a96-35004d7cda82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:48.699839 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:48.771531 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 17:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:48.932385 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d33e8-81cd-4821-aa2c-6f87f14cc7bf None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:48.934528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-424d33e8-81cd-4821-aa2c-6f87f14cc7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:49.077056 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d33e8-81cd-4821-aa2c-6f87f14cc7bf None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:49.116787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:49.190721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef9c30-1479-4205-8b2e-caaeb32d4b14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:49.191233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef9c30-1479-4205-8b2e-caaeb32d4b14 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:49.191506 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef9c30-1479-4205-8b2e-caaeb32d4b14 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:49.191771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef9c30-1479-4205-8b2e-caaeb32d4b14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:49.193404 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-daef9c30-1479-4205-8b2e-caaeb32d4b14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:49.285478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef9c30-1479-4205-8b2e-caaeb32d4b14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:49.287142 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef9c30-1479-4205-8b2e-caaeb32d4b14 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:49.305454 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 709/1419] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:50.197266 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4b226-bbe6-4618-92a1-cfe780ef45b7 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:50.197825 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4b226-bbe6-4618-92a1-cfe780ef45b7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:50.197825 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4b226-bbe6-4618-92a1-cfe780ef45b7 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:50.205715 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 17:07:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:07:50.205830 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896decf-aedf-4423-85a4-513e8f20e2f5 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:50.205830 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896decf-aedf-4423-85a4-513e8f20e2f5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:50.205830 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896decf-aedf-4423-85a4-513e8f20e2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:50.259396 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:50.282215 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c896decf-aedf-4423-85a4-513e8f20e2f5 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:50.283497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c896decf-aedf-4423-85a4-513e8f20e2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:50.378892 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 17:07:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:50.532657 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4b226-bbe6-4618-92a1-cfe780ef45b7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:50.534521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-96e4b226-bbe6-4618-92a1-cfe780ef45b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:50.679954 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4b226-bbe6-4618-92a1-cfe780ef45b7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:50.691933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:50.793063 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620da8e-c2d7-4178-a4fc-b878aecda41a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:50.793701 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620da8e-c2d7-4178-a4fc-b878aecda41a service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:50.793981 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620da8e-c2d7-4178-a4fc-b878aecda41a service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:50.794249 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620da8e-c2d7-4178-a4fc-b878aecda41a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:50.795879 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7620da8e-c2d7-4178-a4fc-b878aecda41a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:50.884516 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7620da8e-c2d7-4178-a4fc-b878aecda41a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:50.886110 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7620da8e-c2d7-4178-a4fc-b878aecda41a service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:50.904641 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 712/1422] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:51.773062 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7ee08-331a-4d47-96b1-c77c09afa041 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:51.773645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7ee08-331a-4d47-96b1-c77c09afa041 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:51.773645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7ee08-331a-4d47-96b1-c77c09afa041 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:51.774983 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 17:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:07:51.780629 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21469f-4e92-4463-b5c1-8f70ba21f77e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:51.781058 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21469f-4e92-4463-b5c1-8f70ba21f77e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:51.781183 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21469f-4e92-4463-b5c1-8f70ba21f77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:51.801490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:51.867148 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-de21469f-4e92-4463-b5c1-8f70ba21f77e None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:51.868281 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-de21469f-4e92-4463-b5c1-8f70ba21f77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:51.972688 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 17:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:52.123657 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7ee08-331a-4d47-96b1-c77c09afa041 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:52.125184 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a9a7ee08-331a-4d47-96b1-c77c09afa041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:52.251640 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:52.292804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7ee08-331a-4d47-96b1-c77c09afa041 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:52.405845 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ea1c0-941f-4052-9888-87c01c240e49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:52.406386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ea1c0-941f-4052-9888-87c01c240e49 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:52.406660 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ea1c0-941f-4052-9888-87c01c240e49 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:52.406930 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ea1c0-941f-4052-9888-87c01c240e49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:52.408530 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bd4ea1c0-941f-4052-9888-87c01c240e49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:52.483991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:52.498034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ea1c0-941f-4052-9888-87c01c240e49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:52.499810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ea1c0-941f-4052-9888-87c01c240e49 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:52.519397 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 712/1425] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:07:53.395843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0ea93-1517-491f-93ff-d0850abcaecb None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:53.396402 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0ea93-1517-491f-93ff-d0850abcaecb None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:53.396402 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0ea93-1517-491f-93ff-d0850abcaecb None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:53.397722 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 17:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:07:53.403477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6980eed-89ff-46e3-a7a4-6c0d9b79c2c3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:53.403917 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6980eed-89ff-46e3-a7a4-6c0d9b79c2c3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:53.404024 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6980eed-89ff-46e3-a7a4-6c0d9b79c2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:53.478917 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f6980eed-89ff-46e3-a7a4-6c0d9b79c2c3 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:53.480868 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f6980eed-89ff-46e3-a7a4-6c0d9b79c2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:53.579154 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 17:07:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:53.732026 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf0ea93-1517-491f-93ff-d0850abcaecb None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:53.733614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-dcf0ea93-1517-491f-93ff-d0850abcaecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:53.821530 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:53.885603 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf0ea93-1517-491f-93ff-d0850abcaecb None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:54.001934 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0381d-1ffd-472c-a900-966a66f51f1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:54.002568 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0381d-1ffd-472c-a900-966a66f51f1a service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:54.002851 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0381d-1ffd-472c-a900-966a66f51f1a service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:54.003123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0381d-1ffd-472c-a900-966a66f51f1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:54.004725 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8fc0381d-1ffd-472c-a900-966a66f51f1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:54.066901 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:54.099028 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc0381d-1ffd-472c-a900-966a66f51f1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:54.100931 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc0381d-1ffd-472c-a900-966a66f51f1a service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:54.120526 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 715/1428] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:55.061357 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff21b2d-dbdf-4571-8233-6d51620691a9 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:55.061898 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff21b2d-dbdf-4571-8233-6d51620691a9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:55.061898 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff21b2d-dbdf-4571-8233-6d51620691a9 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:55.063181 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 17:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:07:55.068592 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0a2f1-ca5e-421e-a9a8-cee7526f86a4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:55.069026 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0a2f1-ca5e-421e-a9a8-cee7526f86a4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:55.075056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0a2f1-ca5e-421e-a9a8-cee7526f86a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:55.147249 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7bd0a2f1-ca5e-421e-a9a8-cee7526f86a4 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:55.148480 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7bd0a2f1-ca5e-421e-a9a8-cee7526f86a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:55.248840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 17:07:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:55.402676 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff21b2d-dbdf-4571-8233-6d51620691a9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:55.404103 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0ff21b2d-dbdf-4571-8233-6d51620691a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:55.453626 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:55.557735 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff21b2d-dbdf-4571-8233-6d51620691a9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:55.674894 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5a61e-06fc-4da0-8ce4-cd169361151a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:55.675411 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5a61e-06fc-4da0-8ce4-cd169361151a service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:55.675684 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5a61e-06fc-4da0-8ce4-cd169361151a service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:55.675953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5a61e-06fc-4da0-8ce4-cd169361151a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:55.677576 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c4b5a61e-06fc-4da0-8ce4-cd169361151a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:55.685341 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:55.770804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5a61e-06fc-4da0-8ce4-cd169361151a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:55.772383 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5a61e-06fc-4da0-8ce4-cd169361151a service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:55.790390 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 715/1431] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:56.702312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9640435-98c0-4de6-b91d-7927af9a02cb None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:56.702888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9640435-98c0-4de6-b91d-7927af9a02cb None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:56.702888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9640435-98c0-4de6-b91d-7927af9a02cb None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:56.704265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 17:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:07:56.709457 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974e439-096a-4ada-815e-a8c0c9a15658 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:56.709901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974e439-096a-4ada-815e-a8c0c9a15658 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:56.710019 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974e439-096a-4ada-815e-a8c0c9a15658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:56.786745 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6974e439-096a-4ada-815e-a8c0c9a15658 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:56.787964 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6974e439-096a-4ada-815e-a8c0c9a15658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:56.856109 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:56.886386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 17:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:57.037664 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9640435-98c0-4de6-b91d-7927af9a02cb None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:57.039225 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a9640435-98c0-4de6-b91d-7927af9a02cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:57.057214 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:57.187217 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9640435-98c0-4de6-b91d-7927af9a02cb None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:57.276758 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:57.303264 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64d51e-19bb-4a40-84d0-4f9124cf3630 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:57.303875 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64d51e-19bb-4a40-84d0-4f9124cf3630 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:57.304153 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64d51e-19bb-4a40-84d0-4f9124cf3630 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:57.304423 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64d51e-19bb-4a40-84d0-4f9124cf3630 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:57.306053 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8c64d51e-19bb-4a40-84d0-4f9124cf3630 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:57.396648 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c64d51e-19bb-4a40-84d0-4f9124cf3630 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:57.398231 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c64d51e-19bb-4a40-84d0-4f9124cf3630 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:57.416065 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 718/1434] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:58.300149 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e437ee-ac68-44c0-8e45-5238a945ff27 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:58.300149 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e437ee-ac68-44c0-8e45-5238a945ff27 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:58.300149 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e437ee-ac68-44c0-8e45-5238a945ff27 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:58.300149 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 17:07:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:07:58.304811 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba35ed8b-6da2-483c-91fe-54ee912fd731 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:58.305223 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba35ed8b-6da2-483c-91fe-54ee912fd731 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:58.305352 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba35ed8b-6da2-483c-91fe-54ee912fd731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:58.383505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ba35ed8b-6da2-483c-91fe-54ee912fd731 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:58.384709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ba35ed8b-6da2-483c-91fe-54ee912fd731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:58.428740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:58.486369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 17:07:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:07:58.639388 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e437ee-ac68-44c0-8e45-5238a945ff27 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:07:58.640910 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e7e437ee-ac68-44c0-8e45-5238a945ff27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:58.795353 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e437ee-ac68-44c0-8e45-5238a945ff27 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:07:58.850370 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:07:58.913061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc363e6-7a61-4a83-a7e8-c348e0884907 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:07:58.913658 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc363e6-7a61-4a83-a7e8-c348e0884907 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:58.913929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc363e6-7a61-4a83-a7e8-c348e0884907 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:58.914193 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc363e6-7a61-4a83-a7e8-c348e0884907 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:58.915839 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fbc363e6-7a61-4a83-a7e8-c348e0884907 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:59.008819 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc363e6-7a61-4a83-a7e8-c348e0884907 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:07:59.010493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc363e6-7a61-4a83-a7e8-c348e0884907 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:07:59.029607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 718/1437] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:07:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:07:59.907716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf2ca3-6b28-4bd6-9128-747a3bdb9f34 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:59.907716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf2ca3-6b28-4bd6-9128-747a3bdb9f34 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:59.907716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf2ca3-6b28-4bd6-9128-747a3bdb9f34 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:59.909014 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 17:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:07:59.914260 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e650551-3265-46a0-9dd8-6a9f6b628024 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:07:59.914707 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e650551-3265-46a0-9dd8-6a9f6b628024 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:07:59.914950 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e650551-3265-46a0-9dd8-6a9f6b628024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:07:59.992409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7e650551-3265-46a0-9dd8-6a9f6b628024 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:07:59.993676 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7e650551-3265-46a0-9dd8-6a9f6b628024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:07:59.995751 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:00.094655 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 17:07:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:00.247828 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf2ca3-6b28-4bd6-9128-747a3bdb9f34 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:00.249351 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2daf2ca3-6b28-4bd6-9128-747a3bdb9f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:00.396181 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf2ca3-6b28-4bd6-9128-747a3bdb9f34 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:00.423029 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:00.514913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c955a7-a2e8-4377-b3d5-b444879e0f3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:00.515422 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c955a7-a2e8-4377-b3d5-b444879e0f3b service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:00.515696 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c955a7-a2e8-4377-b3d5-b444879e0f3b service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:00.515963 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c955a7-a2e8-4377-b3d5-b444879e0f3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:00.517844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b4c955a7-a2e8-4377-b3d5-b444879e0f3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:00.618848 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c955a7-a2e8-4377-b3d5-b444879e0f3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:00.620979 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c955a7-a2e8-4377-b3d5-b444879e0f3b service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:00.641240 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 721/1440] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:02.568016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e83e3-d5a4-49cd-ad39-5bd0e0f8e9d2 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:02.568577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e83e3-d5a4-49cd-ad39-5bd0e0f8e9d2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:02.568577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e83e3-d5a4-49cd-ad39-5bd0e0f8e9d2 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:02.569858 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 17:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:08:02.576000 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ddf22d-c9df-4b79-8b3d-5afac067e31a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:02.576471 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ddf22d-c9df-4b79-8b3d-5afac067e31a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:02.576571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ddf22d-c9df-4b79-8b3d-5afac067e31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:02.620595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:02.647908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-86ddf22d-c9df-4b79-8b3d-5afac067e31a None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:02.649121 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-86ddf22d-c9df-4b79-8b3d-5afac067e31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:02.750523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 17:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:02.789495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e83e3-d5a4-49cd-ad39-5bd0e0f8e9d2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:02.791113 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3a4e83e3-d5a4-49cd-ad39-5bd0e0f8e9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:02.906830 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e83e3-d5a4-49cd-ad39-5bd0e0f8e9d2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:02.941071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:02.983400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf555e-c6cd-4e99-aab1-8edc7b2a9208 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:02.984015 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf555e-c6cd-4e99-aab1-8edc7b2a9208 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:02.984264 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf555e-c6cd-4e99-aab1-8edc7b2a9208 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:02.984529 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf555e-c6cd-4e99-aab1-8edc7b2a9208 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:02.986118 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4ecf555e-c6cd-4e99-aab1-8edc7b2a9208 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:03.094661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf555e-c6cd-4e99-aab1-8edc7b2a9208 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:03.095999 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf555e-c6cd-4e99-aab1-8edc7b2a9208 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:03.114171 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 721/1443] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:05.424370 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92431b55-7514-461e-9bc6-1ad5e76ce2a9 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:05.424982 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92431b55-7514-461e-9bc6-1ad5e76ce2a9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:05.424982 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92431b55-7514-461e-9bc6-1ad5e76ce2a9 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:05.426329 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 17:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:08:05.431878 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468279f-35a9-44b6-8e81-d5a1a0d83fe4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:05.432323 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468279f-35a9-44b6-8e81-d5a1a0d83fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:05.432432 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468279f-35a9-44b6-8e81-d5a1a0d83fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:05.506410 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7468279f-35a9-44b6-8e81-d5a1a0d83fe4 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:05.507723 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7468279f-35a9-44b6-8e81-d5a1a0d83fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:05.524870 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:05.617698 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 17:08:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:05.668912 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92431b55-7514-461e-9bc6-1ad5e76ce2a9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:05.670643 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-92431b55-7514-461e-9bc6-1ad5e76ce2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:05.788894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92431b55-7514-461e-9bc6-1ad5e76ce2a9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:05.803548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:05.865597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d3bf4-ebac-4310-a7b7-8761f6838c74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:05.866204 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d3bf4-ebac-4310-a7b7-8761f6838c74 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:05.866512 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d3bf4-ebac-4310-a7b7-8761f6838c74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:05.866783 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d3bf4-ebac-4310-a7b7-8761f6838c74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:05.868382 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bd8d3bf4-ebac-4310-a7b7-8761f6838c74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:05.978141 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d3bf4-ebac-4310-a7b7-8761f6838c74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:05.979635 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d3bf4-ebac-4310-a7b7-8761f6838c74 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:05.998590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 724/1446] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:08:08.419467 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a63d57-56b2-4e50-b1e5-f575643b6ae3 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:08.420031 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a63d57-56b2-4e50-b1e5-f575643b6ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:08.420031 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a63d57-56b2-4e50-b1e5-f575643b6ae3 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:08.421368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 17:08:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:08:08.426743 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966b757-4396-48e1-a14d-d791eb8d3a99 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:08.427185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966b757-4396-48e1-a14d-d791eb8d3a99 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:08.427288 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966b757-4396-48e1-a14d-d791eb8d3a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:08.476808 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:08.495948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0966b757-4396-48e1-a14d-d791eb8d3a99 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:08.497270 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0966b757-4396-48e1-a14d-d791eb8d3a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:08.596352 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 17:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:08.636953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a63d57-56b2-4e50-b1e5-f575643b6ae3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:08.638594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b9a63d57-56b2-4e50-b1e5-f575643b6ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:08.742453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a63d57-56b2-4e50-b1e5-f575643b6ae3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:08.783350 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:08.818241 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11af18f-bb27-49b3-bd8b-372c90974bc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:08.819147 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11af18f-bb27-49b3-bd8b-372c90974bc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:08.819593 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11af18f-bb27-49b3-bd8b-372c90974bc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:08.820138 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11af18f-bb27-49b3-bd8b-372c90974bc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:08.821752 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d11af18f-bb27-49b3-bd8b-372c90974bc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:08.931236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11af18f-bb27-49b3-bd8b-372c90974bc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:08.932673 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11af18f-bb27-49b3-bd8b-372c90974bc5 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:08.951280 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 724/1449] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:11.170862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5ac80-3d45-49f0-ab72-1a33aacc5c36 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:11.171430 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5ac80-3d45-49f0-ab72-1a33aacc5c36 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:11.171430 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5ac80-3d45-49f0-ab72-1a33aacc5c36 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:11.172759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 17:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:08:11.178006 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a0e93-e03d-42eb-8d57-4c379d98094e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:11.178467 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a0e93-e03d-42eb-8d57-4c379d98094e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:11.178859 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a0e93-e03d-42eb-8d57-4c379d98094e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:11.244013 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-927a0e93-e03d-42eb-8d57-4c379d98094e None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:11.245220 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-927a0e93-e03d-42eb-8d57-4c379d98094e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:11.267146 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:11.344578 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 17:08:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:11.384194 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5ac80-3d45-49f0-ab72-1a33aacc5c36 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:11.385756 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0cd5ac80-3d45-49f0-ab72-1a33aacc5c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:11.491686 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5ac80-3d45-49f0-ab72-1a33aacc5c36 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:11.512125 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:11.565208 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e1a63-72bc-495f-9a2d-695311b67358 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:11.565806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e1a63-72bc-495f-9a2d-695311b67358 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:11.566089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e1a63-72bc-495f-9a2d-695311b67358 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:11.566400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e1a63-72bc-495f-9a2d-695311b67358 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:11.568013 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-259e1a63-72bc-495f-9a2d-695311b67358 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:11.677179 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e1a63-72bc-495f-9a2d-695311b67358 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:11.678640 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e1a63-72bc-495f-9a2d-695311b67358 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:11.696625 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 727/1452] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:13.806934 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079beeaa-ba8f-43fe-a371-1136a70ce197 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:13.807494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079beeaa-ba8f-43fe-a371-1136a70ce197 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:13.807494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079beeaa-ba8f-43fe-a371-1136a70ce197 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:13.808890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 17:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:08:13.814759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30d7ae-3ec1-40cd-9ae7-a6dbf1fb551c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:13.815170 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30d7ae-3ec1-40cd-9ae7-a6dbf1fb551c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:13.815283 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30d7ae-3ec1-40cd-9ae7-a6dbf1fb551c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:13.878556 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:13.881396 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ff30d7ae-3ec1-40cd-9ae7-a6dbf1fb551c None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:13.882654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ff30d7ae-3ec1-40cd-9ae7-a6dbf1fb551c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:13.981399 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 17:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:14.020423 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079beeaa-ba8f-43fe-a371-1136a70ce197 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:14.021960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-079beeaa-ba8f-43fe-a371-1136a70ce197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:14.125528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079beeaa-ba8f-43fe-a371-1136a70ce197 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:14.175077 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:14.199433 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8541e-e454-45ec-b5bf-ea5c55cbb922 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:14.200035 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8541e-e454-45ec-b5bf-ea5c55cbb922 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:14.200309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8541e-e454-45ec-b5bf-ea5c55cbb922 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:14.200574 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8541e-e454-45ec-b5bf-ea5c55cbb922 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:14.202170 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-33b8541e-e454-45ec-b5bf-ea5c55cbb922 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:14.315162 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8541e-e454-45ec-b5bf-ea5c55cbb922 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:14.316593 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8541e-e454-45ec-b5bf-ea5c55cbb922 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:14.336579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 727/1455] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:16.402830 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c38c74-77e8-4ad1-93b0-80ba93bcaf1d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:16.403396 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c38c74-77e8-4ad1-93b0-80ba93bcaf1d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:16.403396 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c38c74-77e8-4ad1-93b0-80ba93bcaf1d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:16.404705 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 17:08:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:08:16.409935 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3da17-3f95-4670-9ca1-56250440570c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:16.410421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3da17-3f95-4670-9ca1-56250440570c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:16.410539 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3da17-3f95-4670-9ca1-56250440570c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:16.475771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-70a3da17-3f95-4670-9ca1-56250440570c None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:16.476976 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-70a3da17-3f95-4670-9ca1-56250440570c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:16.506379 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:16.573661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 17:08:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:16.613047 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c38c74-77e8-4ad1-93b0-80ba93bcaf1d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:16.614639 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-87c38c74-77e8-4ad1-93b0-80ba93bcaf1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:16.717824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c38c74-77e8-4ad1-93b0-80ba93bcaf1d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:16.746276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:16.793603 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06ca88-7e31-4256-affc-c8eaf3e643ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:16.794116 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06ca88-7e31-4256-affc-c8eaf3e643ca service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:16.794426 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06ca88-7e31-4256-affc-c8eaf3e643ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:16.794696 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06ca88-7e31-4256-affc-c8eaf3e643ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:16.796309 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9d06ca88-7e31-4256-affc-c8eaf3e643ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:16.906971 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06ca88-7e31-4256-affc-c8eaf3e643ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:16.908420 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06ca88-7e31-4256-affc-c8eaf3e643ca service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:16.926194 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 730/1458] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:19.038311 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83ac9f-5cec-45e9-b0f8-196a280f285f None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:19.038896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83ac9f-5cec-45e9-b0f8-196a280f285f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:19.038896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83ac9f-5cec-45e9-b0f8-196a280f285f None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:19.040162 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 17:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:08:19.045429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3cdfea-853d-4ca0-a52d-d91f41dd5db1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:19.045885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3cdfea-853d-4ca0-a52d-d91f41dd5db1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:19.046004 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3cdfea-853d-4ca0-a52d-d91f41dd5db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:19.117702 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8a3cdfea-853d-4ca0-a52d-d91f41dd5db1 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:19.119011 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8a3cdfea-853d-4ca0-a52d-d91f41dd5db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:19.127887 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:19.216453 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 17:08:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:19.255226 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83ac9f-5cec-45e9-b0f8-196a280f285f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:19.256770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ee83ac9f-5cec-45e9-b0f8-196a280f285f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:19.362638 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83ac9f-5cec-45e9-b0f8-196a280f285f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:19.419276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:19.437312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650130e-2ca6-4692-8c8f-c98910fcf02e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:19.437931 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650130e-2ca6-4692-8c8f-c98910fcf02e service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:19.438200 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650130e-2ca6-4692-8c8f-c98910fcf02e service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:19.438491 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650130e-2ca6-4692-8c8f-c98910fcf02e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:19.440078 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0650130e-2ca6-4692-8c8f-c98910fcf02e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:19.547029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650130e-2ca6-4692-8c8f-c98910fcf02e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:19.548447 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650130e-2ca6-4692-8c8f-c98910fcf02e service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:19.566492 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 730/1461] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:21.660685 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e98bd-285c-4be5-b47d-393bdbd25088 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:21.661244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e98bd-285c-4be5-b47d-393bdbd25088 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:21.661244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e98bd-285c-4be5-b47d-393bdbd25088 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:21.662582 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 17:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:08:21.668379 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8ab16-6355-42d0-8fc0-5ec5da4fc1ac None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:21.668928 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8ab16-6355-42d0-8fc0-5ec5da4fc1ac None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:21.669037 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8ab16-6355-42d0-8fc0-5ec5da4fc1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:21.734680 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-85d8ab16-6355-42d0-8fc0-5ec5da4fc1ac None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:21.735889 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-85d8ab16-6355-42d0-8fc0-5ec5da4fc1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:21.772813 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:21.829340 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 17:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:21.867194 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e98bd-285c-4be5-b47d-393bdbd25088 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:21.868721 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-633e98bd-285c-4be5-b47d-393bdbd25088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:21.969648 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e98bd-285c-4be5-b47d-393bdbd25088 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:22.006587 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:22.043093 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c0a58-01e6-4e41-a62d-081f1bbd6ae7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:22.043691 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c0a58-01e6-4e41-a62d-081f1bbd6ae7 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:22.043965 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c0a58-01e6-4e41-a62d-081f1bbd6ae7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:22.044229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c0a58-01e6-4e41-a62d-081f1bbd6ae7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:22.045815 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f31c0a58-01e6-4e41-a62d-081f1bbd6ae7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:22.155203 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c0a58-01e6-4e41-a62d-081f1bbd6ae7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:22.156562 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c0a58-01e6-4e41-a62d-081f1bbd6ae7 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:22.175308 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 733/1464] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:24.348525 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ab1dd-cf1a-4787-bbc7-35a3788f967c None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:24.349086 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ab1dd-cf1a-4787-bbc7-35a3788f967c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:24.349086 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ab1dd-cf1a-4787-bbc7-35a3788f967c None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:24.350523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 17:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:08:24.355940 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da791495-a04a-4450-8ab7-b1828a63b175 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:24.356408 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da791495-a04a-4450-8ab7-b1828a63b175 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:24.356502 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da791495-a04a-4450-8ab7-b1828a63b175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:24.425083 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-da791495-a04a-4450-8ab7-b1828a63b175 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:24.426225 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-da791495-a04a-4450-8ab7-b1828a63b175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:24.444821 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:24.525883 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 17:08:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:08:24.564252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ab1dd-cf1a-4787-bbc7-35a3788f967c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:24.565790 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3e9ab1dd-cf1a-4787-bbc7-35a3788f967c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:24.673455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ab1dd-cf1a-4787-bbc7-35a3788f967c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:24.741144 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:24.748251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3afb9-690f-46eb-811a-2af29a96a481 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:24.748851 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3afb9-690f-46eb-811a-2af29a96a481 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:24.753832 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3afb9-690f-46eb-811a-2af29a96a481 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:24.753832 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3afb9-690f-46eb-811a-2af29a96a481 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:24.753832 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-53f3afb9-690f-46eb-811a-2af29a96a481 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:24.861935 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f3afb9-690f-46eb-811a-2af29a96a481 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:24.863408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f3afb9-690f-46eb-811a-2af29a96a481 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:24.881878 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 733/1467] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:26.934893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75722d-9872-4364-a4b8-2afe6323d5fc None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:26.935491 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75722d-9872-4364-a4b8-2afe6323d5fc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:26.935491 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75722d-9872-4364-a4b8-2afe6323d5fc None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:26.936755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 17:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:08:26.941956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d41c44-c364-49ae-830e-73544761a6de None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:26.942342 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d41c44-c364-49ae-830e-73544761a6de None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:26.942548 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d41c44-c364-49ae-830e-73544761a6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:27.008524 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f6d41c44-c364-49ae-830e-73544761a6de None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:27.009764 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f6d41c44-c364-49ae-830e-73544761a6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:27.061763 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:27.109776 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 17:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:27.147397 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75722d-9872-4364-a4b8-2afe6323d5fc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:27.148936 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ff75722d-9872-4364-a4b8-2afe6323d5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:27.252670 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75722d-9872-4364-a4b8-2afe6323d5fc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:27.302126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:27.329373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b9f510-6bb4-4789-99ca-7423b45c0f93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:27.329983 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9f510-6bb4-4789-99ca-7423b45c0f93 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:27.330259 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9f510-6bb4-4789-99ca-7423b45c0f93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:27.330558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9f510-6bb4-4789-99ca-7423b45c0f93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:27.332174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-88b9f510-6bb4-4789-99ca-7423b45c0f93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:27.442697 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b9f510-6bb4-4789-99ca-7423b45c0f93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:27.444141 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b9f510-6bb4-4789-99ca-7423b45c0f93 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:27.464133 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 736/1470] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:37.107970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=22 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:37.461708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=66 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:47.357738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833fac3-8245-4456-a8e3-063d6f4bf689 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:47.358425 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833fac3-8245-4456-a8e3-063d6f4bf689 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:47.358425 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833fac3-8245-4456-a8e3-063d6f4bf689 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:47.359681 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 735/1471] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:08:47.365262 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac5a29-7202-482b-aeaa-513ebd5f84be None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:47.365770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac5a29-7202-482b-aeaa-513ebd5f84be None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:47.365888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac5a29-7202-482b-aeaa-513ebd5f84be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:47.437650 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a7ac5a29-7202-482b-aeaa-513ebd5f84be None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:47.438936 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a7ac5a29-7202-482b-aeaa-513ebd5f84be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:47.538036 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 737/1472] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:47.628133 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d758b-8dfb-41a5-98e8-27b5f48fe75e None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:47.628416 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d758b-8dfb-41a5-98e8-27b5f48fe75e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:47.628714 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d758b-8dfb-41a5-98e8-27b5f48fe75e None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:47.630104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 736/1473] 10.222.0.45 () {54 vars in 967 bytes} [Wed Aug 10 17:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:08:47.636232 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09301ef-9f41-42e3-a987-9d399ea3ac81 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:47.636232 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09301ef-9f41-42e3-a987-9d399ea3ac81 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:47.636232 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09301ef-9f41-42e3-a987-9d399ea3ac81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:47.715773 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a09301ef-9f41-42e3-a987-9d399ea3ac81 None None] MFA Rules not processed for user `10f4f54890624da29ff217081445b6bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:47.717003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a09301ef-9f41-42e3-a987-9d399ea3ac81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:47.739321 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:47.798578 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 738/1474] 10.222.0.45 () {58 vars in 1064 bytes} [Wed Aug 10 17:08:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:47.806927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78d758b-8dfb-41a5-98e8-27b5f48fe75e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:47.809782 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e78d758b-8dfb-41a5-98e8-27b5f48fe75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:47.918304 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78d758b-8dfb-41a5-98e8-27b5f48fe75e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:47.996814 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8eabec-0c84-4b88-a74c-c0a8399a8141 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10f4f54890624da29ff217081445b6bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:47.997410 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8eabec-0c84-4b88-a74c-c0a8399a8141 service glance] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:47.997684 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8eabec-0c84-4b88-a74c-c0a8399a8141 service glance] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:47.997951 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8eabec-0c84-4b88-a74c-c0a8399a8141 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:47.999579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-eb8eabec-0c84-4b88-a74c-c0a8399a8141 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:48.059439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:48.117019 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8eabec-0c84-4b88-a74c-c0a8399a8141 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:48.118525 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8eabec-0c84-4b88-a74c-c0a8399a8141 service glance] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:48.137576 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 737/1475] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:50.281493 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78e4a6-4de6-43bd-9d70-8906540b8885 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:50.282035 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78e4a6-4de6-43bd-9d70-8906540b8885 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:50.282035 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78e4a6-4de6-43bd-9d70-8906540b8885 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:50.283421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 739/1476] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:08:50.288692 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71221c0-5a92-45bb-bab1-09c4bfa0d498 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:50.289148 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71221c0-5a92-45bb-bab1-09c4bfa0d498 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:50.289273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71221c0-5a92-45bb-bab1-09c4bfa0d498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:50.357154 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e71221c0-5a92-45bb-bab1-09c4bfa0d498 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:50.358419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e71221c0-5a92-45bb-bab1-09c4bfa0d498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:50.455895 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 738/1477] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:50.480316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78e4a6-4de6-43bd-9d70-8906540b8885 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:50.481855 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2d78e4a6-4de6-43bd-9d70-8906540b8885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:50.608543 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78e4a6-4de6-43bd-9d70-8906540b8885 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:50.661212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:50.705878 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95be29-ce7d-4ecd-b81a-956d6521694c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:50.706513 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95be29-ce7d-4ecd-b81a-956d6521694c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:50.706794 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95be29-ce7d-4ecd-b81a-956d6521694c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:50.707065 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95be29-ce7d-4ecd-b81a-956d6521694c admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:50.707698 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95be29-ce7d-4ecd-b81a-956d6521694c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:50.709479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-fe95be29-ce7d-4ecd-b81a-956d6521694c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 17:08:50.709479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Aug 10 17:08:50.709479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: Aug 10 17:08:50.710645 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95be29-ce7d-4ecd-b81a-956d6521694c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:50.730847 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 740/1478] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 10 17:08:50 2022] GET /identity/v3/projects => generated 2553 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:08:52.774983 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d426b7-98a0-49fa-a520-3db1cd7d85da None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:52.775543 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d426b7-98a0-49fa-a520-3db1cd7d85da None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:52.775543 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d426b7-98a0-49fa-a520-3db1cd7d85da None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:52.777344 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 739/1479] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:08:52.782539 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82513c38-43d9-422c-86b1-0b4835546a6e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:52.782955 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82513c38-43d9-422c-86b1-0b4835546a6e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:52.783094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82513c38-43d9-422c-86b1-0b4835546a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:52.849148 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-82513c38-43d9-422c-86b1-0b4835546a6e None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:52.850385 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-82513c38-43d9-422c-86b1-0b4835546a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:52.949501 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 741/1480] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:08:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:52.992908 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d426b7-98a0-49fa-a520-3db1cd7d85da None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:52.994465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-70d426b7-98a0-49fa-a520-3db1cd7d85da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:53.004780 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:53.140337 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d426b7-98a0-49fa-a520-3db1cd7d85da None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:53.220963 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:53.254043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51205270-0977-4108-9c47-22fe0b45cf22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:53.254705 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51205270-0977-4108-9c47-22fe0b45cf22 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:53.254980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51205270-0977-4108-9c47-22fe0b45cf22 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:53.255249 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51205270-0977-4108-9c47-22fe0b45cf22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:53.256863 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-51205270-0977-4108-9c47-22fe0b45cf22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:53.347415 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51205270-0977-4108-9c47-22fe0b45cf22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:53.349036 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51205270-0977-4108-9c47-22fe0b45cf22 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:53.367775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 740/1481] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:55.591444 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5db97b-b872-41ed-a2db-f67d10b7584a None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:55.592021 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5db97b-b872-41ed-a2db-f67d10b7584a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:55.592021 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5db97b-b872-41ed-a2db-f67d10b7584a None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:55.593330 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 742/1482] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:08:55.598709 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20615754-1ed1-46aa-8656-fe4620ad4fe3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:55.599178 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20615754-1ed1-46aa-8656-fe4620ad4fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:55.599289 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20615754-1ed1-46aa-8656-fe4620ad4fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:55.667666 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-20615754-1ed1-46aa-8656-fe4620ad4fe3 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:55.668894 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-20615754-1ed1-46aa-8656-fe4620ad4fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:55.714583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:55.768149 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 741/1483] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:08:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:08:55.804780 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5db97b-b872-41ed-a2db-f67d10b7584a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:55.806364 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2e5db97b-b872-41ed-a2db-f67d10b7584a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:55.831572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:55.913538 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5db97b-b872-41ed-a2db-f67d10b7584a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:55.987178 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb7042-4f07-4a62-80a5-cd4dfbc9f284 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:55.987783 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb7042-4f07-4a62-80a5-cd4dfbc9f284 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:55.988059 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb7042-4f07-4a62-80a5-cd4dfbc9f284 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:55.988325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb7042-4f07-4a62-80a5-cd4dfbc9f284 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:55.989948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-defb7042-4f07-4a62-80a5-cd4dfbc9f284 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:56.064858 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:56.102577 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb7042-4f07-4a62-80a5-cd4dfbc9f284 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:56.104225 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb7042-4f07-4a62-80a5-cd4dfbc9f284 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:56.125690 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 743/1484] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:08:58.234991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475021e-965f-46fe-9ca3-f1631c6049b0 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:58.235546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475021e-965f-46fe-9ca3-f1631c6049b0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:58.235546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475021e-965f-46fe-9ca3-f1631c6049b0 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:58.236900 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 742/1485] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:08:58.242204 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f575ffb-29f6-4c5a-b296-d4742ddbada9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:58.242657 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f575ffb-29f6-4c5a-b296-d4742ddbada9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:58.242801 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f575ffb-29f6-4c5a-b296-d4742ddbada9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:58.312769 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4f575ffb-29f6-4c5a-b296-d4742ddbada9 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:08:58.314296 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4f575ffb-29f6-4c5a-b296-d4742ddbada9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:58.412897 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 744/1486] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:08:58.452240 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8475021e-965f-46fe-9ca3-f1631c6049b0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:08:58.453791 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8475021e-965f-46fe-9ca3-f1631c6049b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:58.561968 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8475021e-965f-46fe-9ca3-f1631c6049b0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:08:58.635772 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:08:58.637566 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb9afa9-a8a6-4c8b-9062-bd28875e0e27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:08:58.638161 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9afa9-a8a6-4c8b-9062-bd28875e0e27 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:08:58.638478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9afa9-a8a6-4c8b-9062-bd28875e0e27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:08:58.638748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9afa9-a8a6-4c8b-9062-bd28875e0e27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:08:58.640356 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9bb9afa9-a8a6-4c8b-9062-bd28875e0e27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:08:58.750016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb9afa9-a8a6-4c8b-9062-bd28875e0e27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:08:58.751406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb9afa9-a8a6-4c8b-9062-bd28875e0e27 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:08:58.770162 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 743/1487] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:08:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:09:00.948984 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de58da-daa0-45dd-872a-99029c52c0f2 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:09:00.949508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de58da-daa0-45dd-872a-99029c52c0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:09:00.949508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de58da-daa0-45dd-872a-99029c52c0f2 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:09:00.950837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 745/1488] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:09:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 17:09:00.956812 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07c259-472f-4b5d-844f-2491e05aaaec None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:09:00.957248 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07c259-472f-4b5d-844f-2491e05aaaec None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:09:00.957354 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07c259-472f-4b5d-844f-2491e05aaaec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:09:01.023852 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1c07c259-472f-4b5d-844f-2491e05aaaec None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:09:01.025068 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1c07c259-472f-4b5d-844f-2491e05aaaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:09:01.078843 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:09:01.124972 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 744/1489] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:09:01.165136 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de58da-daa0-45dd-872a-99029c52c0f2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:09:01.167442 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-90de58da-daa0-45dd-872a-99029c52c0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:09:01.203333 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:09:01.273945 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de58da-daa0-45dd-872a-99029c52c0f2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:09:01.347494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae607d-09c4-445b-9817-c23f9e55cde2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:09:01.348089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae607d-09c4-445b-9817-c23f9e55cde2 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:09:01.348362 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae607d-09c4-445b-9817-c23f9e55cde2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:09:01.348627 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae607d-09c4-445b-9817-c23f9e55cde2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:09:01.350243 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1cae607d-09c4-445b-9817-c23f9e55cde2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:09:01.431305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:09:01.461641 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae607d-09c4-445b-9817-c23f9e55cde2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:09:01.463054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae607d-09c4-445b-9817-c23f9e55cde2 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:09:01.482893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 746/1490] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:09:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:09:03.912402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85821f0d-a2f8-4951-95ed-a79f8bd6c7b6 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:09:03.912934 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85821f0d-a2f8-4951-95ed-a79f8bd6c7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:09:03.912934 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85821f0d-a2f8-4951-95ed-a79f8bd6c7b6 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:09:03.914200 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 745/1491] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 10 17:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:09:03.920084 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abea7c-11b5-4b48-bbeb-448ea30af7a2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:09:03.920566 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abea7c-11b5-4b48-bbeb-448ea30af7a2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:09:03.920666 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abea7c-11b5-4b48-bbeb-448ea30af7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:09:03.985527 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-85abea7c-11b5-4b48-bbeb-448ea30af7a2 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:09:03.986778 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-85abea7c-11b5-4b48-bbeb-448ea30af7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:09:04.082710 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 747/1492] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 10 17:09:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:09:04.121972 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85821f0d-a2f8-4951-95ed-a79f8bd6c7b6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:09:04.123558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-85821f0d-a2f8-4951-95ed-a79f8bd6c7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:09:04.226426 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85821f0d-a2f8-4951-95ed-a79f8bd6c7b6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:09:04.302126 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f7e30-6ca8-48ee-bb16-138df7fc3474 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:09:04.307091 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f7e30-6ca8-48ee-bb16-138df7fc3474 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:09:04.307091 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f7e30-6ca8-48ee-bb16-138df7fc3474 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:09:04.307091 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f7e30-6ca8-48ee-bb16-138df7fc3474 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:09:04.307091 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-974f7e30-6ca8-48ee-bb16-138df7fc3474 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:09:04.314384 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:09:04.415727 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974f7e30-6ca8-48ee-bb16-138df7fc3474 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:09:04.417712 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974f7e30-6ca8-48ee-bb16-138df7fc3474 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:09:04.436039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 746/1493] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:09:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:09:14.080871 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=82 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:09:14.433772 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=50 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:10:25.613100 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6b432-19d0-4d72-b57b-873250ae5eee None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:10:25.613556 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6b432-19d0-4d72-b57b-873250ae5eee None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:10:25.613652 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6b432-19d0-4d72-b57b-873250ae5eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:10:25.689701 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-acb6b432-19d0-4d72-b57b-873250ae5eee None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:10:25.690969 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-acb6b432-19d0-4d72-b57b-873250ae5eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:10:25.788774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 748/1494] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:10:25.797574 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20119d-ae45-47b4-9c93-1780a43ac9d8 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:10:25.798096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20119d-ae45-47b4-9c93-1780a43ac9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:10:25.798096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20119d-ae45-47b4-9c93-1780a43ac9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:10:25.875585 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-bf20119d-ae45-47b4-9c93-1780a43ac9d8 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:10:25.876745 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bf20119d-ae45-47b4-9c93-1780a43ac9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:10:25.971014 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 747/1495] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:10:25.975743 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6b432-19d0-4d72-b57b-873250ae5eee None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:10:25.976953 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-acb6b432-19d0-4d72-b57b-873250ae5eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:10:25.988939 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6b432-19d0-4d72-b57b-873250ae5eee None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:10:26.049703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:10:26.083992 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b914256-4827-4ce8-830e-50c15ca8cd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:10:26.084591 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b914256-4827-4ce8-830e-50c15ca8cd5b admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:10:26.084862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b914256-4827-4ce8-830e-50c15ca8cd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:10:26.085128 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b914256-4827-4ce8-830e-50c15ca8cd5b admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:10:26.085755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b914256-4827-4ce8-830e-50c15ca8cd5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:10:26.086049 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b914256-4827-4ce8-830e-50c15ca8cd5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:10:26.087094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-1b914256-4827-4ce8-830e-50c15ca8cd5b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 17:10:26.087094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Aug 10 17:10:26.087094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: Aug 10 17:10:26.087988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b914256-4827-4ce8-830e-50c15ca8cd5b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:10:26.100101 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 749/1496] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:10:26.108370 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20119d-ae45-47b4-9c93-1780a43ac9d8 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:10:26.117455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20119d-ae45-47b4-9c93-1780a43ac9d8 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:10:26.179118 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:10:26.222096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbcb78-6540-4d15-b25a-af966816ae0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:10:26.222715 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbcb78-6540-4d15-b25a-af966816ae0b admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:10:26.222984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbcb78-6540-4d15-b25a-af966816ae0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:10:26.223246 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbcb78-6540-4d15-b25a-af966816ae0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:10:26.223868 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccbcb78-6540-4d15-b25a-af966816ae0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:10:26.225583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-0ccbcb78-6540-4d15-b25a-af966816ae0b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 17:10:26.225583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Aug 10 17:10:26.225583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: Aug 10 17:10:26.226794 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccbcb78-6540-4d15-b25a-af966816ae0b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:10:26.339612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 748/1497] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:10:26 2022] POST /identity/v3/projects => generated 377 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:10:26.343554 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b914256-4827-4ce8-830e-50c15ca8cd5b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:10:26.345158 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1b914256-4827-4ce8-830e-50c15ca8cd5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:10:26.381307 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b914256-4827-4ce8-830e-50c15ca8cd5b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:10:26.476376 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96d570-d9db-4d17-aabe-552afd821670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:10:26.476947 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96d570-d9db-4d17-aabe-552afd821670 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:10:26.477165 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96d570-d9db-4d17-aabe-552afd821670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:10:26.477429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96d570-d9db-4d17-aabe-552afd821670 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:10:26.478042 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e96d570-d9db-4d17-aabe-552afd821670 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:10:26.479136 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-3e96d570-d9db-4d17-aabe-552afd821670 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 17:10:26.479136 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Aug 10 17:10:26.479136 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: Aug 10 17:10:26.480324 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e96d570-d9db-4d17-aabe-552afd821670 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:10:26.579286 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 750/1498] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:10:26 2022] POST /identity/v3/users => generated 289 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:10:26.583788 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbcb78-6540-4d15-b25a-af966816ae0b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:10:26.585346 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0ccbcb78-6540-4d15-b25a-af966816ae0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:10:26.647953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbcb78-6540-4d15-b25a-af966816ae0b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:10:26.730527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:10:26.740954 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25705f-e334-44b7-9a1e-2bb7006c953e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:10:26.741606 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25705f-e334-44b7-9a1e-2bb7006c953e admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:10:26.741869 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25705f-e334-44b7-9a1e-2bb7006c953e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:10:26.742127 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25705f-e334-44b7-9a1e-2bb7006c953e admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:10:26.742798 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25705f-e334-44b7-9a1e-2bb7006c953e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:10:26.743796 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-4f25705f-e334-44b7-9a1e-2bb7006c953e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 17:10:26.743796 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Aug 10 17:10:26.743796 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: Aug 10 17:10:26.745517 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25705f-e334-44b7-9a1e-2bb7006c953e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:10:26.758561 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 749/1499] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:10:26.763212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96d570-d9db-4d17-aabe-552afd821670 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:10:26.764737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3e96d570-d9db-4d17-aabe-552afd821670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:10:26.777361 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96d570-d9db-4d17-aabe-552afd821670 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:10:26.803399 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:10:26.877471 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6a7ba-4564-40db-a20f-b33dff9cc9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:10:26.878072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6a7ba-4564-40db-a20f-b33dff9cc9aa admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:10:26.878343 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6a7ba-4564-40db-a20f-b33dff9cc9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:10:26.878627 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6a7ba-4564-40db-a20f-b33dff9cc9aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:10:26.879269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6a7ba-4564-40db-a20f-b33dff9cc9aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:10:26.880276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-dbb6a7ba-4564-40db-a20f-b33dff9cc9aa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 17:10:26.880276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Aug 10 17:10:26.880276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: Aug 10 17:10:26.881156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6a7ba-4564-40db-a20f-b33dff9cc9aa admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:10:26.894917 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 751/1500] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:10:26.899062 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25705f-e334-44b7-9a1e-2bb7006c953e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:10:26.900581 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4f25705f-e334-44b7-9a1e-2bb7006c953e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:10:26.933652 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25705f-e334-44b7-9a1e-2bb7006c953e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:10:27.029003 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733a21a-fc11-4893-9436-cc465ece694e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:10:27.029566 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733a21a-fc11-4893-9436-cc465ece694e admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:10:27.029845 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733a21a-fc11-4893-9436-cc465ece694e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:10:27.030113 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733a21a-fc11-4893-9436-cc465ece694e admin admin] PATH_INFO: `/v3/projects/0b263f803869416bb9f0052007f454c9/users/8ec51846a389459cb0a7490265f9a0b5/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:10:27.052897 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0733a21a-fc11-4893-9436-cc465ece694e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b263f803869416bb9f0052007f454c9, user_id=8ec51846a389459cb0a7490265f9a0b5, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:10:27.053932 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-0733a21a-fc11-4893-9436-cc465ece694e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 17:10:27.053932 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Aug 10 17:10:27.053932 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: Aug 10 17:10:27.055924 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0733a21a-fc11-4893-9436-cc465ece694e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:10:27.143658 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 750/1501] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:10:26 2022] PUT /identity/v3/projects/0b263f803869416bb9f0052007f454c9/users/8ec51846a389459cb0a7490265f9a0b5/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:10:27.156111 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ffc72-68ed-4edc-8ebe-989d434b84cd None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:10:27.156403 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ffc72-68ed-4edc-8ebe-989d434b84cd None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:10:27.156683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ffc72-68ed-4edc-8ebe-989d434b84cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:10:27.220416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-df1ffc72-68ed-4edc-8ebe-989d434b84cd None None] MFA Rules not processed for user `8ec51846a389459cb0a7490265f9a0b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:10:27.221646 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-df1ffc72-68ed-4edc-8ebe-989d434b84cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:10:27.305326 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 752/1502] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:10:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:10:27.318640 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402304c-ad41-4184-b331-d69c84c7bffc None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:10:27.318928 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402304c-ad41-4184-b331-d69c84c7bffc None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:10:27.319206 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402304c-ad41-4184-b331-d69c84c7bffc None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:10:27.320565 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 751/1503] 10.222.0.45 () {54 vars in 964 bytes} [Wed Aug 10 17:10:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:10:27.327215 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3f586-34b4-4b31-b1c8-261cfba4ff50 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:10:27.327506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3f586-34b4-4b31-b1c8-261cfba4ff50 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:10:27.327782 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3f586-34b4-4b31-b1c8-261cfba4ff50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:10:27.360193 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:10:27.406984 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9ba3f586-34b4-4b31-b1c8-261cfba4ff50 None None] MFA Rules not processed for user `56d5c4afb9844fbbbe699f958deae6e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:10:27.408219 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9ba3f586-34b4-4b31-b1c8-261cfba4ff50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:10:27.612403 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 753/1504] 10.222.0.45 () {58 vars in 1061 bytes} [Wed Aug 10 17:10:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:10:27.621252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402304c-ad41-4184-b331-d69c84c7bffc None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:10:27.623539 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0402304c-ad41-4184-b331-d69c84c7bffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:10:27.677318 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:10:27.731627 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402304c-ad41-4184-b331-d69c84c7bffc None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:10:27.806322 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39385dc9-3a0a-44f8-ab0a-1c3fbd55ed39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:10:27.807477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39385dc9-3a0a-44f8-ab0a-1c3fbd55ed39 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:10:27.807927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39385dc9-3a0a-44f8-ab0a-1c3fbd55ed39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:10:27.808383 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39385dc9-3a0a-44f8-ab0a-1c3fbd55ed39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:10:27.810333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-39385dc9-3a0a-44f8-ab0a-1c3fbd55ed39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:10:27.908267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39385dc9-3a0a-44f8-ab0a-1c3fbd55ed39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:10:27.909594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39385dc9-3a0a-44f8-ab0a-1c3fbd55ed39 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:10:27.913049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:10:27.927081 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 752/1505] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:10:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:10:27.972693 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74ca0a-0e0d-46df-bebd-d860e78bff2b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:10:27.973126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74ca0a-0e0d-46df-bebd-d860e78bff2b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:10:27.973233 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74ca0a-0e0d-46df-bebd-d860e78bff2b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:10:27.974491 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 754/1506] 10.222.0.45 () {52 vars in 830 bytes} [Wed Aug 10 17:10:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:10:27.978063 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39385dc9-3a0a-44f8-ab0a-1c3fbd55ed39 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:10:27.979891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-39385dc9-3a0a-44f8-ab0a-1c3fbd55ed39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:10:28.077164 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39385dc9-3a0a-44f8-ab0a-1c3fbd55ed39 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:10:28.170088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9cf19-2693-47f3-88ec-23e03d157adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:10:28.170653 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9cf19-2693-47f3-88ec-23e03d157adf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:10:28.170928 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9cf19-2693-47f3-88ec-23e03d157adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:10:28.171195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9cf19-2693-47f3-88ec-23e03d157adf admin admin] PATH_INFO: `/v3/users/8ec51846a389459cb0a7490265f9a0b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:10:28.184073 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a9cf19-2693-47f3-88ec-23e03d157adf admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ec51846a389459cb0a7490265f9a0b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:10:28.185122 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-f0a9cf19-2693-47f3-88ec-23e03d157adf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 17:10:28.185122 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Aug 10 17:10:28.185122 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: Aug 10 17:10:28.186327 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a9cf19-2693-47f3-88ec-23e03d157adf admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:10:28.196216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:10:28.386229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f0a9cf19-2693-47f3-88ec-23e03d157adf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ec51846a389459cb0a7490265f9a0b5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:10:28.472926 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f0a9cf19-2693-47f3-88ec-23e03d157adf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ec51846a389459cb0a7490265f9a0b5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:10:28.497730 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f0a9cf19-2693-47f3-88ec-23e03d157adf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ec51846a389459cb0a7490265f9a0b5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:10:28.498307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f0a9cf19-2693-47f3-88ec-23e03d157adf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ec51846a389459cb0a7490265f9a0b5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:10:28.498754 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f0a9cf19-2693-47f3-88ec-23e03d157adf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ec51846a389459cb0a7490265f9a0b5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:10:28.520131 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 753/1507] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:10:27 2022] DELETE /identity/v3/users/8ec51846a389459cb0a7490265f9a0b5 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:10:28.523675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46f92d-335c-4b5d-851c-db708a2054f7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:10:28.523959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46f92d-335c-4b5d-851c-db708a2054f7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:10:28.524258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46f92d-335c-4b5d-851c-db708a2054f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:10:28.581075 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3e46f92d-335c-4b5d-851c-db708a2054f7 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:10:28.582297 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3e46f92d-335c-4b5d-851c-db708a2054f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:10:28.658567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:10:28.671631 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 755/1508] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:10:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:10:28.681485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9cf19-2693-47f3-88ec-23e03d157adf admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:10:28.683718 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f0a9cf19-2693-47f3-88ec-23e03d157adf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:10:28.792531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9cf19-2693-47f3-88ec-23e03d157adf admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:10:28.851669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:10:28.872957 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3bd87a-cfc7-4e53-9e8b-ac838dc819ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:10:28.873592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bd87a-cfc7-4e53-9e8b-ac838dc819ff service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:10:28.873868 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bd87a-cfc7-4e53-9e8b-ac838dc819ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:10:28.874136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bd87a-cfc7-4e53-9e8b-ac838dc819ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:10:28.875814 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4c3bd87a-cfc7-4e53-9e8b-ac838dc819ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:10:28.986548 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3bd87a-cfc7-4e53-9e8b-ac838dc819ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:10:28.988005 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3bd87a-cfc7-4e53-9e8b-ac838dc819ff service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:10:29.007001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 754/1509] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:10:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:10:29.465236 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46f92d-335c-4b5d-851c-db708a2054f7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:10:29.466538 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3e46f92d-335c-4b5d-851c-db708a2054f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:10:29.478067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46f92d-335c-4b5d-851c-db708a2054f7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:10:29.580346 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0cb4cf-dc21-4d27-b301-e8dfcd85d312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:10:29.581038 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0cb4cf-dc21-4d27-b301-e8dfcd85d312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:10:29.581316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0cb4cf-dc21-4d27-b301-e8dfcd85d312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:10:29.581586 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0cb4cf-dc21-4d27-b301-e8dfcd85d312 admin admin] PATH_INFO: `/v3/projects/0b263f803869416bb9f0052007f454c9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:10:29.592397 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0cb4cf-dc21-4d27-b301-e8dfcd85d312 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b263f803869416bb9f0052007f454c9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:10:29.593436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-cc0cb4cf-dc21-4d27-b301-e8dfcd85d312 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 17:10:29.593436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Aug 10 17:10:29.593436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: Aug 10 17:10:29.594652 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0cb4cf-dc21-4d27-b301-e8dfcd85d312 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:10:29.707183 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cc0cb4cf-dc21-4d27-b301-e8dfcd85d312 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b263f803869416bb9f0052007f454c9', 'request_id': 'req-cc0cb4cf-dc21-4d27-b301-e8dfcd85d312'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:10:29.722240 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cc0cb4cf-dc21-4d27-b301-e8dfcd85d312 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b263f803869416bb9f0052007f454c9', 'request_id': 'req-cc0cb4cf-dc21-4d27-b301-e8dfcd85d312'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:10:29.799770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cc0cb4cf-dc21-4d27-b301-e8dfcd85d312 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b263f803869416bb9f0052007f454c9', 'request_id': 'req-cc0cb4cf-dc21-4d27-b301-e8dfcd85d312'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:10:29.824101 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cc0cb4cf-dc21-4d27-b301-e8dfcd85d312 admin admin] The token cache is being invalidate because project 0b263f803869416bb9f0052007f454c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:10:29.824390 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cc0cb4cf-dc21-4d27-b301-e8dfcd85d312 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:10:29.824820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cc0cb4cf-dc21-4d27-b301-e8dfcd85d312 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:10:29.826410 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 756/1510] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:10:29 2022] DELETE /identity/v3/projects/0b263f803869416bb9f0052007f454c9 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:10:39.004766 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=62 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:10:39.822019 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:12:05.397468 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa276d-f208-4c34-af21-22ae51e471e1 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:05.398106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa276d-f208-4c34-af21-22ae51e471e1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:05.398106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa276d-f208-4c34-af21-22ae51e471e1 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:05.399533 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 17:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:12:05.405971 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f2a31-c456-4c9a-a81f-06050d666403 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:05.405971 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f2a31-c456-4c9a-a81f-06050d666403 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:05.405971 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f2a31-c456-4c9a-a81f-06050d666403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:05.492023 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2e4f2a31-c456-4c9a-a81f-06050d666403 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:12:05.493290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2e4f2a31-c456-4c9a-a81f-06050d666403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:12:05.600383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 17:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:12:05.759303 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa276d-f208-4c34-af21-22ae51e471e1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:12:05.761432 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-27aa276d-f208-4c34-af21-22ae51e471e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:12:05.879939 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa276d-f208-4c34-af21-22ae51e471e1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:12:05.885572 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c004c-f1c6-491e-aed1-f258db654eb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:12:05.886179 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c004c-f1c6-491e-aed1-f258db654eb7 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:05.886488 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c004c-f1c6-491e-aed1-f258db654eb7 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:05.886762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c004c-f1c6-491e-aed1-f258db654eb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:05.895523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541c004c-f1c6-491e-aed1-f258db654eb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:12:05.897103 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541c004c-f1c6-491e-aed1-f258db654eb7 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:12:05.907357 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 756/1513] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:12:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:12:06.485045 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8753b9-d29d-4466-92e3-f79b43636772 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:06.485591 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8753b9-d29d-4466-92e3-f79b43636772 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:06.486019 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8753b9-d29d-4466-92e3-f79b43636772 None None] PATH_INFO: `/v3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:06.487764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 758/1514] 10.222.0.49 () {54 vars in 925 bytes} [Wed Aug 10 17:12:06 2022] GET /identity/v3 => generated 255 bytes in 5 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:12:06.494567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e816d8d-2e7e-4e18-9422-9a4c62db7b90 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:06.495037 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e816d8d-2e7e-4e18-9422-9a4c62db7b90 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:06.495404 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e816d8d-2e7e-4e18-9422-9a4c62db7b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:06.560201 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3e816d8d-2e7e-4e18-9422-9a4c62db7b90 None None] MFA Rules not processed for user `8cd39d9a583a4d73a5654017be106019`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:12:06.561432 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3e816d8d-2e7e-4e18-9422-9a4c62db7b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:12:06.637659 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:12:06.643558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 757/1515] 10.222.0.49 () {58 vars in 1015 bytes} [Wed Aug 10 17:12:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:12:06.754813 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc72652-65c3-4994-83f3-fefb33456d91 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:06.755098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc72652-65c3-4994-83f3-fefb33456d91 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:06.755369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc72652-65c3-4994-83f3-fefb33456d91 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:06.756761 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 759/1516] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 17:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:12:06.762477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2f0fe-0973-4a8c-ad61-2bb25d78977c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:06.762760 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2f0fe-0973-4a8c-ad61-2bb25d78977c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:06.763036 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2f0fe-0973-4a8c-ad61-2bb25d78977c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:06.782893 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d2a2f0fe-0973-4a8c-ad61-2bb25d78977c None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:12:06.784153 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d2a2f0fe-0973-4a8c-ad61-2bb25d78977c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:12:06.792360 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 758/1517] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 17:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:12:06.948146 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc72652-65c3-4994-83f3-fefb33456d91 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:12:06.963260 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc72652-65c3-4994-83f3-fefb33456d91 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:12:06.969580 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f10cc-06bb-4bc6-98f8-d31eaaa39bff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:12:06.970200 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f10cc-06bb-4bc6-98f8-d31eaaa39bff service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:06.970511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f10cc-06bb-4bc6-98f8-d31eaaa39bff service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:06.970782 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f10cc-06bb-4bc6-98f8-d31eaaa39bff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:06.980121 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f10cc-06bb-4bc6-98f8-d31eaaa39bff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:12:06.981761 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f10cc-06bb-4bc6-98f8-d31eaaa39bff service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:12:06.992447 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 760/1518] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:12:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:12:15.828484 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4671af6-fd53-42b3-be8d-7e1917fcfaa0 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:15.829048 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4671af6-fd53-42b3-be8d-7e1917fcfaa0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:15.829048 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4671af6-fd53-42b3-be8d-7e1917fcfaa0 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:15.830272 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 759/1519] 10.222.0.45 () {54 vars in 961 bytes} [Wed Aug 10 17:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:12:15.835474 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908db63-2050-4bcd-9543-12b14c0a737e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:15.835928 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908db63-2050-4bcd-9543-12b14c0a737e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:15.836042 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908db63-2050-4bcd-9543-12b14c0a737e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:15.856096 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5908db63-2050-4bcd-9543-12b14c0a737e None None] MFA Rules not processed for user `8cd39d9a583a4d73a5654017be106019`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:12:15.857290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5908db63-2050-4bcd-9543-12b14c0a737e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:12:15.865812 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 761/1520] 10.222.0.45 () {58 vars in 1058 bytes} [Wed Aug 10 17:12:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:12:15.873844 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4671af6-fd53-42b3-be8d-7e1917fcfaa0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:12:15.887982 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4671af6-fd53-42b3-be8d-7e1917fcfaa0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:12:15.893355 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb16e2e-be31-4448-b353-da590cd9689c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cd39d9a583a4d73a5654017be106019', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:12:15.893947 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb16e2e-be31-4448-b353-da590cd9689c service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:15.894226 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb16e2e-be31-4448-b353-da590cd9689c service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:15.894534 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb16e2e-be31-4448-b353-da590cd9689c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:15.903686 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb16e2e-be31-4448-b353-da590cd9689c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:12:15.908402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb16e2e-be31-4448-b353-da590cd9689c service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:12:15.915431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 760/1521] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:12:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:12:18.241607 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c3a87-34c3-4213-a1b0-22617285f6f2 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:18.242199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c3a87-34c3-4213-a1b0-22617285f6f2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:18.242199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c3a87-34c3-4213-a1b0-22617285f6f2 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:18.243582 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 17:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:12:18.248636 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8029dbb-5e49-4cdd-8554-80e9c0a53808 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:18.249099 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8029dbb-5e49-4cdd-8554-80e9c0a53808 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:18.249209 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8029dbb-5e49-4cdd-8554-80e9c0a53808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:18.269147 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f8029dbb-5e49-4cdd-8554-80e9c0a53808 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:12:18.270365 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f8029dbb-5e49-4cdd-8554-80e9c0a53808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:12:18.279021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 17:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:12:18.435072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c3a87-34c3-4213-a1b0-22617285f6f2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:12:18.449756 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c3a87-34c3-4213-a1b0-22617285f6f2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:12:18.455191 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd5fad-cd50-4854-8232-85c5da49a566 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:12:18.460646 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd5fad-cd50-4854-8232-85c5da49a566 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:18.460646 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd5fad-cd50-4854-8232-85c5da49a566 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:18.460646 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd5fad-cd50-4854-8232-85c5da49a566 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:18.464799 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd5fad-cd50-4854-8232-85c5da49a566 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:12:18.466457 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd5fad-cd50-4854-8232-85c5da49a566 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:12:18.476106 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 763/1524] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:12:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:12:19.309923 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee733ac9-10b7-4fdc-8b46-e840f590a967 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:19.310539 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee733ac9-10b7-4fdc-8b46-e840f590a967 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:19.310539 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee733ac9-10b7-4fdc-8b46-e840f590a967 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:19.311852 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 17:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:12:19.316889 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383b65b-002e-4fa2-8322-90c58489536a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:19.317330 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383b65b-002e-4fa2-8322-90c58489536a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:19.317437 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383b65b-002e-4fa2-8322-90c58489536a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:19.337022 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1383b65b-002e-4fa2-8322-90c58489536a None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:12:19.338211 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1383b65b-002e-4fa2-8322-90c58489536a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:12:19.347190 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 17:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:12:19.500915 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee733ac9-10b7-4fdc-8b46-e840f590a967 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:12:19.515785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee733ac9-10b7-4fdc-8b46-e840f590a967 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:12:19.521286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe9e3b-3659-4914-a279-5338d6d1302c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:12:19.521913 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe9e3b-3659-4914-a279-5338d6d1302c service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:19.522189 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe9e3b-3659-4914-a279-5338d6d1302c service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:19.522503 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe9e3b-3659-4914-a279-5338d6d1302c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:19.531501 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe9e3b-3659-4914-a279-5338d6d1302c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:12:19.533072 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe9e3b-3659-4914-a279-5338d6d1302c service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:12:19.543308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 763/1527] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:12:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:12:23.592026 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59dd74f-0a03-4eab-982a-e62e4f841743 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:23.592574 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59dd74f-0a03-4eab-982a-e62e4f841743 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:23.592574 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59dd74f-0a03-4eab-982a-e62e4f841743 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:23.593893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 17:12:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:12:23.599248 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2b8e6-142d-4cdd-9202-c2558fd2f1fc None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:23.599688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2b8e6-142d-4cdd-9202-c2558fd2f1fc None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:23.599824 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2b8e6-142d-4cdd-9202-c2558fd2f1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:23.619920 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c0a2b8e6-142d-4cdd-9202-c2558fd2f1fc None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:12:23.621763 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c0a2b8e6-142d-4cdd-9202-c2558fd2f1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:12:23.629666 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 17:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:12:23.707036 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59dd74f-0a03-4eab-982a-e62e4f841743 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:12:23.708708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c59dd74f-0a03-4eab-982a-e62e4f841743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:12:23.743296 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:12:23.792030 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59dd74f-0a03-4eab-982a-e62e4f841743 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:12:23.796959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaaa1d4-438a-49ef-97e9-7af98047a508 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:12:23.797553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaaa1d4-438a-49ef-97e9-7af98047a508 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:23.797825 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaaa1d4-438a-49ef-97e9-7af98047a508 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:23.798089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaaa1d4-438a-49ef-97e9-7af98047a508 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:23.807034 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaaa1d4-438a-49ef-97e9-7af98047a508 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:12:23.808993 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaaa1d4-438a-49ef-97e9-7af98047a508 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:12:23.818481 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 766/1530] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:12:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:12:25.836242 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50813a0-479f-4697-95a2-d90209d2a562 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:25.837043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50813a0-479f-4697-95a2-d90209d2a562 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:25.837043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50813a0-479f-4697-95a2-d90209d2a562 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:25.838160 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 17:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:12:25.843893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde035c5-0edc-4b07-8a72-1b9699817a6a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:25.844445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde035c5-0edc-4b07-8a72-1b9699817a6a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:25.844445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde035c5-0edc-4b07-8a72-1b9699817a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:25.865502 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-dde035c5-0edc-4b07-8a72-1b9699817a6a None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:12:25.866677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-dde035c5-0edc-4b07-8a72-1b9699817a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:12:25.875501 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 17:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:12:25.912708 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50813a0-479f-4697-95a2-d90209d2a562 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:12:25.923565 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50813a0-479f-4697-95a2-d90209d2a562 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:12:25.927082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73c05f-cac3-423d-b417-70e0e256b8e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:12:25.927671 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73c05f-cac3-423d-b417-70e0e256b8e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:25.927961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73c05f-cac3-423d-b417-70e0e256b8e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:25.928218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73c05f-cac3-423d-b417-70e0e256b8e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:25.937188 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73c05f-cac3-423d-b417-70e0e256b8e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:12:25.938637 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73c05f-cac3-423d-b417-70e0e256b8e1 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:12:25.947971 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 766/1533] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:12:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:12:29.547819 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee70111-967b-4ae0-bb20-72c6f5aa9a9f None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:29.548402 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee70111-967b-4ae0-bb20-72c6f5aa9a9f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:29.548513 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee70111-967b-4ae0-bb20-72c6f5aa9a9f None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:29.549886 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 17:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:12:29.555544 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08ca99-d621-40b1-abfb-5d89654b2e27 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:29.556048 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08ca99-d621-40b1-abfb-5d89654b2e27 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:29.556156 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08ca99-d621-40b1-abfb-5d89654b2e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:29.575929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6a08ca99-d621-40b1-abfb-5d89654b2e27 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:12:29.577129 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6a08ca99-d621-40b1-abfb-5d89654b2e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:12:29.585028 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 17:12:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:12:29.896520 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee70111-967b-4ae0-bb20-72c6f5aa9a9f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:12:29.898417 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1ee70111-967b-4ae0-bb20-72c6f5aa9a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:12:29.979607 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee70111-967b-4ae0-bb20-72c6f5aa9a9f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:12:29.983359 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43323640-5c24-48ae-b436-8853706881da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:12:29.983960 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43323640-5c24-48ae-b436-8853706881da service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:29.984266 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43323640-5c24-48ae-b436-8853706881da service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:29.984501 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43323640-5c24-48ae-b436-8853706881da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:29.993602 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43323640-5c24-48ae-b436-8853706881da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:12:29.995150 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43323640-5c24-48ae-b436-8853706881da service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:12:30.005472 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 769/1536] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:12:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:12:32.083964 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e52d34-4aff-4ca7-959c-f633f35a1783 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:32.084547 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e52d34-4aff-4ca7-959c-f633f35a1783 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:32.084547 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e52d34-4aff-4ca7-959c-f633f35a1783 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:32.085864 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 17:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:12:32.091491 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fb499-1ac3-4487-bc24-4746cd05a22c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:32.091932 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fb499-1ac3-4487-bc24-4746cd05a22c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:32.092070 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fb499-1ac3-4487-bc24-4746cd05a22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:32.112573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-544fb499-1ac3-4487-bc24-4746cd05a22c None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:12:32.113760 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-544fb499-1ac3-4487-bc24-4746cd05a22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:12:32.122217 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 17:12:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:12:32.427312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e52d34-4aff-4ca7-959c-f633f35a1783 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:12:32.429000 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-27e52d34-4aff-4ca7-959c-f633f35a1783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:12:32.442628 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e52d34-4aff-4ca7-959c-f633f35a1783 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:12:32.446178 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60663c3-b9f4-4dca-9958-8e1a75eb50e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:12:32.446813 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60663c3-b9f4-4dca-9958-8e1a75eb50e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:12:32.447091 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60663c3-b9f4-4dca-9958-8e1a75eb50e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:12:32.447365 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60663c3-b9f4-4dca-9958-8e1a75eb50e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:12:32.457229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60663c3-b9f4-4dca-9958-8e1a75eb50e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:12:32.458693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60663c3-b9f4-4dca-9958-8e1a75eb50e1 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:12:32.469221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 769/1539] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:12:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:12:42.107737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=63 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:12:42.463938 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=37 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:13:57.584746 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d326e-4876-4f43-a3e1-956a96214e0a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:57.608461 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d326e-4876-4f43-a3e1-956a96214e0a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:57.608461 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d326e-4876-4f43-a3e1-956a96214e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:57.623627 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f90d326e-4876-4f43-a3e1-956a96214e0a None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:13:57.625046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f90d326e-4876-4f43-a3e1-956a96214e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:13:57.642608 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 771/1540] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:13:57.653608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38913da-0910-4ee6-84d4-f6bfc8e2046e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:57.655227 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38913da-0910-4ee6-84d4-f6bfc8e2046e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:57.656001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38913da-0910-4ee6-84d4-f6bfc8e2046e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:57.701229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d38913da-0910-4ee6-84d4-f6bfc8e2046e None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:13:57.704303 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d38913da-0910-4ee6-84d4-f6bfc8e2046e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:13:57.727450 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 770/1541] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:13:57.732679 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d326e-4876-4f43-a3e1-956a96214e0a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:13:57.749780 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d326e-4876-4f43-a3e1-956a96214e0a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:13:57.758255 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f9e8f-d0a1-4ea1-8fc1-c4767e7a8690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:13:57.759086 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f9e8f-d0a1-4ea1-8fc1-c4767e7a8690 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:57.759369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f9e8f-d0a1-4ea1-8fc1-c4767e7a8690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:57.759636 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f9e8f-d0a1-4ea1-8fc1-c4767e7a8690 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:57.760453 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f9e8f-d0a1-4ea1-8fc1-c4767e7a8690 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:13:57.760800 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f9e8f-d0a1-4ea1-8fc1-c4767e7a8690 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:13:57.762611 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f9e8f-d0a1-4ea1-8fc1-c4767e7a8690 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:13:57.779084 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 772/1542] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:13:57.788639 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38913da-0910-4ee6-84d4-f6bfc8e2046e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:13:57.814016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38913da-0910-4ee6-84d4-f6bfc8e2046e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:13:57.833999 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ff1df-1be2-4717-be8a-aaac743fcbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:13:57.846326 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ff1df-1be2-4717-be8a-aaac743fcbe4 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:57.846326 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ff1df-1be2-4717-be8a-aaac743fcbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:57.846326 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ff1df-1be2-4717-be8a-aaac743fcbe4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:57.846326 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ff1df-1be2-4717-be8a-aaac743fcbe4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:13:57.846326 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ff1df-1be2-4717-be8a-aaac743fcbe4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:13:58.034254 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 771/1543] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:13:57 2022] POST /identity/v3/projects => generated 373 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:13:58.038580 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f9e8f-d0a1-4ea1-8fc1-c4767e7a8690 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:13:58.055182 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f9e8f-d0a1-4ea1-8fc1-c4767e7a8690 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:13:58.156467 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99535668-595b-4b55-9661-b1479a96e392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:13:58.157133 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99535668-595b-4b55-9661-b1479a96e392 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:58.157319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99535668-595b-4b55-9661-b1479a96e392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:58.157506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99535668-595b-4b55-9661-b1479a96e392 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:58.158146 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99535668-595b-4b55-9661-b1479a96e392 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:13:58.160293 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99535668-595b-4b55-9661-b1479a96e392 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:13:58.227354 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d950a99-f874-4a4b-bba9-17055ea57572 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:58.227829 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d950a99-f874-4a4b-bba9-17055ea57572 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:58.228201 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d950a99-f874-4a4b-bba9-17055ea57572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:58.279786 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4d950a99-f874-4a4b-bba9-17055ea57572 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:13:58.281767 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4d950a99-f874-4a4b-bba9-17055ea57572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:13:58.297544 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 772/1544] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:13:58.299159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb1d17-d5a0-4f59-9cf1-badd1063f04b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:58.299480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb1d17-d5a0-4f59-9cf1-badd1063f04b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:58.299796 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb1d17-d5a0-4f59-9cf1-badd1063f04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:58.312555 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 773/1545] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:13:58 2022] POST /identity/v3/users => generated 287 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:13:58.319726 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3b816-4fd3-4b4e-8e64-6c96b5a3d999 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:58.320173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3b816-4fd3-4b4e-8e64-6c96b5a3d999 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:58.320352 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3b816-4fd3-4b4e-8e64-6c96b5a3d999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:58.336452 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-32fb1d17-d5a0-4f59-9cf1-badd1063f04b None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:13:58.336768 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-32fb1d17-d5a0-4f59-9cf1-badd1063f04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:13:58.355629 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-68d3b816-4fd3-4b4e-8e64-6c96b5a3d999 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:13:58.357016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-68d3b816-4fd3-4b4e-8e64-6c96b5a3d999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:13:58.357667 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 773/1546] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:13:58.358323 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb1d17-d5a0-4f59-9cf1-badd1063f04b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:13:58.372988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 774/1547] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:13:58.374281 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7047827-321a-413e-8a06-a46c3d2a350d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:58.374644 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7047827-321a-413e-8a06-a46c3d2a350d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:58.374972 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7047827-321a-413e-8a06-a46c3d2a350d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:58.378041 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb1d17-d5a0-4f59-9cf1-badd1063f04b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:13:58.407486 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b85560-806d-4add-a7ef-61b6f073cee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:13:58.408035 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b85560-806d-4add-a7ef-61b6f073cee7 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:58.408306 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b85560-806d-4add-a7ef-61b6f073cee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:58.408590 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b85560-806d-4add-a7ef-61b6f073cee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:58.409245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b85560-806d-4add-a7ef-61b6f073cee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:13:58.411125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b85560-806d-4add-a7ef-61b6f073cee7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:13:58.424542 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c7047827-321a-413e-8a06-a46c3d2a350d None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:13:58.443530 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c7047827-321a-413e-8a06-a46c3d2a350d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:13:58.451922 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 774/1548] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:13:58.451922 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5f9a6-5be7-4b28-9b7c-75a34d339776 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:58.451922 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5f9a6-5be7-4b28-9b7c-75a34d339776 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:58.451922 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5f9a6-5be7-4b28-9b7c-75a34d339776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:58.453134 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 775/1549] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:13:58.453134 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7047827-321a-413e-8a06-a46c3d2a350d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:13:58.462268 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7047827-321a-413e-8a06-a46c3d2a350d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:13:58.480702 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd3dad-f891-4683-9edd-b0d5024c895c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:13:58.482229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd3dad-f891-4683-9edd-b0d5024c895c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:58.486059 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd3dad-f891-4683-9edd-b0d5024c895c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:58.487060 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd3dad-f891-4683-9edd-b0d5024c895c admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:58.488658 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd3dad-f891-4683-9edd-b0d5024c895c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:13:58.492303 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd3dad-f891-4683-9edd-b0d5024c895c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:13:58.498469 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd3dad-f891-4683-9edd-b0d5024c895c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:13:58.508410 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-75f5f9a6-5be7-4b28-9b7c-75a34d339776 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:13:58.515436 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-75f5f9a6-5be7-4b28-9b7c-75a34d339776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:13:58.552273 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 776/1550] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:13:58.552930 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 775/1551] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:13:58.561428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd3dad-f891-4683-9edd-b0d5024c895c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:13:58.562824 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7df46b-0f2a-4d91-a790-1b95cde3cd92 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:58.563866 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7df46b-0f2a-4d91-a790-1b95cde3cd92 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:58.565339 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7df46b-0f2a-4d91-a790-1b95cde3cd92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:58.594894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd3dad-f891-4683-9edd-b0d5024c895c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:13:58.614333 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf4f2d-010e-4219-a073-ff1389ebc1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:13:58.619272 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf4f2d-010e-4219-a073-ff1389ebc1a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:58.619272 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf4f2d-010e-4219-a073-ff1389ebc1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:58.619272 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf4f2d-010e-4219-a073-ff1389ebc1a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:58.622474 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf4f2d-010e-4219-a073-ff1389ebc1a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:13:58.627959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf4f2d-010e-4219-a073-ff1389ebc1a1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:13:58.634869 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0a7df46b-0f2a-4d91-a790-1b95cde3cd92 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:13:58.638927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0a7df46b-0f2a-4d91-a790-1b95cde3cd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:13:58.678368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 776/1552] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:13:58.679561 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 777/1553] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:13:58.679735 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b81599-4f6d-4f81-84e9-a0a28d341c90 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:58.679894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf4f2d-010e-4219-a073-ff1389ebc1a1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:13:58.680060 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b81599-4f6d-4f81-84e9-a0a28d341c90 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:58.680060 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b81599-4f6d-4f81-84e9-a0a28d341c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:58.696367 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf4f2d-010e-4219-a073-ff1389ebc1a1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:13:58.707424 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2dede3-7a07-493c-a546-c73b88fb2637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:13:58.707424 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2dede3-7a07-493c-a546-c73b88fb2637 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:58.707424 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2dede3-7a07-493c-a546-c73b88fb2637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:58.707424 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2dede3-7a07-493c-a546-c73b88fb2637 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:58.708073 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2dede3-7a07-493c-a546-c73b88fb2637 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:13:58.708073 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2dede3-7a07-493c-a546-c73b88fb2637 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:13:58.709894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2dede3-7a07-493c-a546-c73b88fb2637 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:13:58.713784 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e1b81599-4f6d-4f81-84e9-a0a28d341c90 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:13:58.716563 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e1b81599-4f6d-4f81-84e9-a0a28d341c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:13:58.734383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 778/1554] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:13:58.734701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 777/1555] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:13:58.734862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23006408-72c5-48f0-8d38-0a85c66146dc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:58.735072 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b81599-4f6d-4f81-84e9-a0a28d341c90 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:13:58.735229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23006408-72c5-48f0-8d38-0a85c66146dc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:58.735229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23006408-72c5-48f0-8d38-0a85c66146dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:58.759569 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b81599-4f6d-4f81-84e9-a0a28d341c90 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:13:58.765745 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af38b7e-15ab-47d6-ae62-df5d8a9e0092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:13:58.766702 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af38b7e-15ab-47d6-ae62-df5d8a9e0092 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:58.767085 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af38b7e-15ab-47d6-ae62-df5d8a9e0092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:58.768231 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af38b7e-15ab-47d6-ae62-df5d8a9e0092 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:58.768621 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af38b7e-15ab-47d6-ae62-df5d8a9e0092 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:13:58.771256 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af38b7e-15ab-47d6-ae62-df5d8a9e0092 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:13:58.777713 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-23006408-72c5-48f0-8d38-0a85c66146dc None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:13:58.780076 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-23006408-72c5-48f0-8d38-0a85c66146dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:13:58.801861 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 779/1556] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:13:58.803365 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23006408-72c5-48f0-8d38-0a85c66146dc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:13:58.819760 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23006408-72c5-48f0-8d38-0a85c66146dc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:13:58.827787 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5446c06-2d91-42ab-baed-9829811865eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:13:58.828806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5446c06-2d91-42ab-baed-9829811865eb admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:58.829298 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5446c06-2d91-42ab-baed-9829811865eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:58.829780 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5446c06-2d91-42ab-baed-9829811865eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:58.830936 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5446c06-2d91-42ab-baed-9829811865eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:13:58.831449 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5446c06-2d91-42ab-baed-9829811865eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:13:58.834403 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5446c06-2d91-42ab-baed-9829811865eb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:13:58.849830 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 780/1557] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:13:58.850991 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5446c06-2d91-42ab-baed-9829811865eb admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:13:58.863812 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5446c06-2d91-42ab-baed-9829811865eb admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:13:58.871215 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32872c88-cac3-42de-a4eb-91b453c785b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:13:58.872122 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32872c88-cac3-42de-a4eb-91b453c785b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:58.872495 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32872c88-cac3-42de-a4eb-91b453c785b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:58.872856 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32872c88-cac3-42de-a4eb-91b453c785b5 admin admin] PATH_INFO: `/v3/projects/651ed58528604503882d8c2fa3313a73/users/e08cf8c5e0a0497d80afae2f02a24972/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:58.876723 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 778/1558] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:13:58 2022] POST /identity/v3/projects => generated 379 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:13:58.879254 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f96d3f-1ab7-43b5-97a8-c31955c1c101 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:58.879728 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f96d3f-1ab7-43b5-97a8-c31955c1c101 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:58.880169 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f96d3f-1ab7-43b5-97a8-c31955c1c101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:58.899093 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32872c88-cac3-42de-a4eb-91b453c785b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=651ed58528604503882d8c2fa3313a73, user_id=e08cf8c5e0a0497d80afae2f02a24972, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:13:58.900908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-32872c88-cac3-42de-a4eb-91b453c785b5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 17:13:58.900908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Aug 10 17:13:58.900908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: Aug 10 17:13:58.904248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32872c88-cac3-42de-a4eb-91b453c785b5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:13:58.911362 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-44f96d3f-1ab7-43b5-97a8-c31955c1c101 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:13:58.912652 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-44f96d3f-1ab7-43b5-97a8-c31955c1c101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:13:58.981324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 779/1559] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:13:58.982461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f96d3f-1ab7-43b5-97a8-c31955c1c101 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:13:58.989997 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 781/1560] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:13:58 2022] PUT /identity/v3/projects/651ed58528604503882d8c2fa3313a73/users/e08cf8c5e0a0497d80afae2f02a24972/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:13:58.995975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8b1d6-5563-435b-aa2b-bcd25eaa1108 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:58.995975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8b1d6-5563-435b-aa2b-bcd25eaa1108 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:58.995975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8b1d6-5563-435b-aa2b-bcd25eaa1108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:59.000297 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f96d3f-1ab7-43b5-97a8-c31955c1c101 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:13:59.043667 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b4e8b1d6-5563-435b-aa2b-bcd25eaa1108 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:13:59.045700 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b4e8b1d6-5563-435b-aa2b-bcd25eaa1108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:13:59.079255 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:13:59.086126 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675b532-2fb2-4df8-942a-258d80fe957c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:13:59.086852 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675b532-2fb2-4df8-942a-258d80fe957c admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:59.087272 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675b532-2fb2-4df8-942a-258d80fe957c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:59.087712 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675b532-2fb2-4df8-942a-258d80fe957c admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:59.088436 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5675b532-2fb2-4df8-942a-258d80fe957c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:13:59.090871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5675b532-2fb2-4df8-942a-258d80fe957c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:13:59.091525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:13:59.141819 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 782/1561] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:13:59.144381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015a076-c103-4e8f-995b-c7b2c2202db7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:59.144944 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015a076-c103-4e8f-995b-c7b2c2202db7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:59.145496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015a076-c103-4e8f-995b-c7b2c2202db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:59.178198 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7015a076-c103-4e8f-995b-c7b2c2202db7 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:13:59.180318 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7015a076-c103-4e8f-995b-c7b2c2202db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:13:59.192279 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 783/1562] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:13:59.193752 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4c1b7-0d9e-4a0a-8e11-0f8e730f7678 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:59.194158 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4c1b7-0d9e-4a0a-8e11-0f8e730f7678 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:59.194569 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4c1b7-0d9e-4a0a-8e11-0f8e730f7678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:59.214321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 780/1563] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:13:58 2022] POST /identity/v3/projects => generated 393 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:13:59.214321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675b532-2fb2-4df8-942a-258d80fe957c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:13:59.222495 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4df4c1b7-0d9e-4a0a-8e11-0f8e730f7678 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:13:59.224694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4df4c1b7-0d9e-4a0a-8e11-0f8e730f7678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:13:59.300906 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675b532-2fb2-4df8-942a-258d80fe957c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:13:59.306283 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12cb47c-c9f6-40ee-b60a-6d682758e989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:13:59.307152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cb47c-c9f6-40ee-b60a-6d682758e989 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:59.307541 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cb47c-c9f6-40ee-b60a-6d682758e989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:59.308534 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cb47c-c9f6-40ee-b60a-6d682758e989 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:59.309364 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12cb47c-c9f6-40ee-b60a-6d682758e989 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:13:59.310540 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 784/1564] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:13:59.311773 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12cb47c-c9f6-40ee-b60a-6d682758e989 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:13:59.315754 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4c1b7-0d9e-4a0a-8e11-0f8e730f7678 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:13:59.331815 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4c1b7-0d9e-4a0a-8e11-0f8e730f7678 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:13:59.337727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf42be-1131-414b-812f-790dcc01e5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:13:59.338559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf42be-1131-414b-812f-790dcc01e5ec admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:59.339525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf42be-1131-414b-812f-790dcc01e5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:59.339952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf42be-1131-414b-812f-790dcc01e5ec admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:59.340858 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf42be-1131-414b-812f-790dcc01e5ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:13:59.343053 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf42be-1131-414b-812f-790dcc01e5ec admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:13:59.405815 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 781/1565] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:13:59 2022] POST /identity/v3/projects => generated 379 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:13:59.409751 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517b2ca-aea7-4fdb-8724-b2776a783a2f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:59.409751 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517b2ca-aea7-4fdb-8724-b2776a783a2f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:59.409751 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517b2ca-aea7-4fdb-8724-b2776a783a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:59.431216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b517b2ca-aea7-4fdb-8724-b2776a783a2f None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:13:59.432458 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b517b2ca-aea7-4fdb-8724-b2776a783a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:13:59.440405 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 785/1566] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:13:59 2022] POST /identity/v3/users => generated 290 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:13:59.442444 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ae591-994b-400d-a0ea-f0197a31c587 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:59.442830 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ae591-994b-400d-a0ea-f0197a31c587 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:59.443260 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ae591-994b-400d-a0ea-f0197a31c587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:59.478185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-109ae591-994b-400d-a0ea-f0197a31c587 None None] MFA Rules not processed for user `e08cf8c5e0a0497d80afae2f02a24972`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:13:59.480434 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-109ae591-994b-400d-a0ea-f0197a31c587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:13:59.516238 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 782/1567] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:13:59.517445 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517b2ca-aea7-4fdb-8724-b2776a783a2f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:13:59.529082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517b2ca-aea7-4fdb-8724-b2776a783a2f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:13:59.534603 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007ae88-d0fe-4766-a75e-53173ecc2f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:13:59.535139 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007ae88-d0fe-4766-a75e-53173ecc2f83 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:59.535416 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007ae88-d0fe-4766-a75e-53173ecc2f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:59.535680 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007ae88-d0fe-4766-a75e-53173ecc2f83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:59.536332 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007ae88-d0fe-4766-a75e-53173ecc2f83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:13:59.536635 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007ae88-d0fe-4766-a75e-53173ecc2f83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:13:59.537680 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-5007ae88-d0fe-4766-a75e-53173ecc2f83 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 17:13:59.537680 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Aug 10 17:13:59.537680 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: Aug 10 17:13:59.538622 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007ae88-d0fe-4766-a75e-53173ecc2f83 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:13:59.556140 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 783/1568] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:13:59.558769 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b82cd-48c4-42b6-923c-76c5a69ca468 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:59.559262 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b82cd-48c4-42b6-923c-76c5a69ca468 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:59.559738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b82cd-48c4-42b6-923c-76c5a69ca468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:59.587314 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-324b82cd-48c4-42b6-923c-76c5a69ca468 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:13:59.590044 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-324b82cd-48c4-42b6-923c-76c5a69ca468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:13:59.600538 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 786/1569] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:13:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:13:59.600538 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ae591-994b-400d-a0ea-f0197a31c587 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:13:59.609890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 784/1570] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:13:59.609890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b82cd-48c4-42b6-923c-76c5a69ca468 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:13:59.613110 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ae591-994b-400d-a0ea-f0197a31c587 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:13:59.620170 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fbd26-1208-4742-8384-3b7f39755021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:13:59.621092 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fbd26-1208-4742-8384-3b7f39755021 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:59.621741 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fbd26-1208-4742-8384-3b7f39755021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:59.622201 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fbd26-1208-4742-8384-3b7f39755021 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:59.623609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660fbd26-1208-4742-8384-3b7f39755021 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:13:59.624738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b82cd-48c4-42b6-923c-76c5a69ca468 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:13:59.627236 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660fbd26-1208-4742-8384-3b7f39755021 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:13:59.632264 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4879849-6b8c-4471-a9d0-008f069c89da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:13:59.633199 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4879849-6b8c-4471-a9d0-008f069c89da admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:59.633589 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4879849-6b8c-4471-a9d0-008f069c89da admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:59.633969 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4879849-6b8c-4471-a9d0-008f069c89da admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:59.635056 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4879849-6b8c-4471-a9d0-008f069c89da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:13:59.635531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4879849-6b8c-4471-a9d0-008f069c89da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:13:59.639734 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4879849-6b8c-4471-a9d0-008f069c89da admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:13:59.659449 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 785/1571] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:13:59.659449 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4879849-6b8c-4471-a9d0-008f069c89da admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:13:59.675076 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4879849-6b8c-4471-a9d0-008f069c89da admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:13:59.680369 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa2027-9865-40e8-b2d6-14a4a7b137c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:13:59.681187 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa2027-9865-40e8-b2d6-14a4a7b137c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:59.681333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa2027-9865-40e8-b2d6-14a4a7b137c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:59.681491 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa2027-9865-40e8-b2d6-14a4a7b137c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:59.682129 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa2027-9865-40e8-b2d6-14a4a7b137c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:13:59.683774 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-b2fa2027-9865-40e8-b2d6-14a4a7b137c2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 17:13:59.683774 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Aug 10 17:13:59.683774 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: Aug 10 17:13:59.684996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa2027-9865-40e8-b2d6-14a4a7b137c2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:13:59.763848 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 787/1572] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:13:59 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:13:59.765443 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fbd26-1208-4742-8384-3b7f39755021 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:13:59.778199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fbd26-1208-4742-8384-3b7f39755021 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:13:59.783426 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b2379-39a0-404f-bad0-0145f60c0cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:13:59.784323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b2379-39a0-404f-bad0-0145f60c0cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:59.784792 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b2379-39a0-404f-bad0-0145f60c0cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:59.785231 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b2379-39a0-404f-bad0-0145f60c0cd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:59.786277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6b2379-39a0-404f-bad0-0145f60c0cd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:13:59.788942 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6b2379-39a0-404f-bad0-0145f60c0cd3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:13:59.797521 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 786/1573] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:13:59 2022] POST /identity/v3/users => generated 290 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:13:59.798940 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa2027-9865-40e8-b2d6-14a4a7b137c2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:13:59.809420 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 788/1574] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:13:59.810789 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b2379-39a0-404f-bad0-0145f60c0cd3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:13:59.820152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa2027-9865-40e8-b2d6-14a4a7b137c2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:13:59.826336 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b2379-39a0-404f-bad0-0145f60c0cd3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:13:59.827260 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d8a3d0-5d79-4b43-858d-c483927373d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:13:59.828026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8a3d0-5d79-4b43-858d-c483927373d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:59.828439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8a3d0-5d79-4b43-858d-c483927373d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:59.828870 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8a3d0-5d79-4b43-858d-c483927373d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:59.829729 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d8a3d0-5d79-4b43-858d-c483927373d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:13:59.830271 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d8a3d0-5d79-4b43-858d-c483927373d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:13:59.832417 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d8a3d0-5d79-4b43-858d-c483927373d8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:13:59.832670 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb293bf-a9ce-4706-a4b1-3866ba11ce77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:13:59.833278 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb293bf-a9ce-4706-a4b1-3866ba11ce77 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:59.833618 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb293bf-a9ce-4706-a4b1-3866ba11ce77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:59.833948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb293bf-a9ce-4706-a4b1-3866ba11ce77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:59.834769 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb293bf-a9ce-4706-a4b1-3866ba11ce77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:13:59.836316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-cbb293bf-a9ce-4706-a4b1-3866ba11ce77 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 17:13:59.836316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Aug 10 17:13:59.836316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: Aug 10 17:13:59.837934 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb293bf-a9ce-4706-a4b1-3866ba11ce77 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:13:59.849613 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 787/1575] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:13:59.850946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d8a3d0-5d79-4b43-858d-c483927373d8 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:13:59.861477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:13:59.862176 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:13:59.866778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d8a3d0-5d79-4b43-858d-c483927373d8 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:13:59.874408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e944b1-c267-446c-8ba4-8bdbf8287756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:13:59.875179 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e944b1-c267-446c-8ba4-8bdbf8287756 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:59.875546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e944b1-c267-446c-8ba4-8bdbf8287756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:59.875977 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e944b1-c267-446c-8ba4-8bdbf8287756 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:59.876675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e944b1-c267-446c-8ba4-8bdbf8287756 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:13:59.877039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e944b1-c267-446c-8ba4-8bdbf8287756 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:13:59.879014 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e944b1-c267-446c-8ba4-8bdbf8287756 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:13:59.895544 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 788/1576] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:13:59.896359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94b671-9df7-4a94-abaf-973ff8d7bce5 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:59.896789 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94b671-9df7-4a94-abaf-973ff8d7bce5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:59.897125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94b671-9df7-4a94-abaf-973ff8d7bce5 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:59.898591 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 789/1577] 10.222.0.45 () {54 vars in 962 bytes} [Wed Aug 10 17:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:13:59.899577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94b671-9df7-4a94-abaf-973ff8d7bce5 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:13:59.915729 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94b671-9df7-4a94-abaf-973ff8d7bce5 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:13:59.921394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8617b12b-983f-4e13-9e27-e75af008e232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:13:59.922099 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617b12b-983f-4e13-9e27-e75af008e232 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:13:59.922530 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617b12b-983f-4e13-9e27-e75af008e232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:13:59.922851 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617b12b-983f-4e13-9e27-e75af008e232 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:13:59.923747 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8617b12b-983f-4e13-9e27-e75af008e232 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:13:59.925830 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8617b12b-983f-4e13-9e27-e75af008e232 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:13:59.981664 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 789/1578] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:13:59 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:13:59.983113 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb293bf-a9ce-4706-a4b1-3866ba11ce77 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:00.053803 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 790/1579] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:13:59 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:00.055179 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8617b12b-983f-4e13-9e27-e75af008e232 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:00.061283 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb293bf-a9ce-4706-a4b1-3866ba11ce77 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:00.136318 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23abdd2a-287c-45b2-8181-bfa53ca8d0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:00.137735 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abdd2a-287c-45b2-8181-bfa53ca8d0ba admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:00.138490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abdd2a-287c-45b2-8181-bfa53ca8d0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:00.138821 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abdd2a-287c-45b2-8181-bfa53ca8d0ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:00.139704 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23abdd2a-287c-45b2-8181-bfa53ca8d0ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:00.141946 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23abdd2a-287c-45b2-8181-bfa53ca8d0ba admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:00.143446 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8617b12b-983f-4e13-9e27-e75af008e232 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:00.151713 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff336f1-6da2-4900-8d21-d4b155682e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:00.152682 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff336f1-6da2-4900-8d21-d4b155682e37 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:00.156553 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff336f1-6da2-4900-8d21-d4b155682e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:00.156553 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff336f1-6da2-4900-8d21-d4b155682e37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:00.156553 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff336f1-6da2-4900-8d21-d4b155682e37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:00.157401 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff336f1-6da2-4900-8d21-d4b155682e37 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:00.159718 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 790/1580] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:00.161243 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23abdd2a-287c-45b2-8181-bfa53ca8d0ba admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:00.173494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 791/1581] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:00.174740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff336f1-6da2-4900-8d21-d4b155682e37 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:00.178061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23abdd2a-287c-45b2-8181-bfa53ca8d0ba admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:00.184952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b912c86-1df0-49b8-91cd-989dd6e7a6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:00.185668 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b912c86-1df0-49b8-91cd-989dd6e7a6ee admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:00.186011 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b912c86-1df0-49b8-91cd-989dd6e7a6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:00.186388 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b912c86-1df0-49b8-91cd-989dd6e7a6ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:00.187158 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b912c86-1df0-49b8-91cd-989dd6e7a6ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:00.187849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff336f1-6da2-4900-8d21-d4b155682e37 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:00.189276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b912c86-1df0-49b8-91cd-989dd6e7a6ee admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:00.193892 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce5c44-386b-40b1-83a8-6207d21ad13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:00.194703 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce5c44-386b-40b1-83a8-6207d21ad13a admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:00.195082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce5c44-386b-40b1-83a8-6207d21ad13a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:00.195457 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce5c44-386b-40b1-83a8-6207d21ad13a admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:00.196241 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce5c44-386b-40b1-83a8-6207d21ad13a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:00.199191 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce5c44-386b-40b1-83a8-6207d21ad13a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:00.202212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 791/1582] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:00.203533 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b912c86-1df0-49b8-91cd-989dd6e7a6ee admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:00.215819 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b912c86-1df0-49b8-91cd-989dd6e7a6ee admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:00.222515 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f513fb-5005-41e8-8ae7-b1015bbc21e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:00.223550 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f513fb-5005-41e8-8ae7-b1015bbc21e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:00.224117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f513fb-5005-41e8-8ae7-b1015bbc21e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:00.224439 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f513fb-5005-41e8-8ae7-b1015bbc21e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:00.225142 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f513fb-5005-41e8-8ae7-b1015bbc21e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:00.227234 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f513fb-5005-41e8-8ae7-b1015bbc21e6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:00.336576 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 792/1583] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:14:00 2022] POST /identity/v3/projects => generated 387 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:00.351323 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36325d4-79b9-480c-975a-c72a4f60b2c0 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:00.351323 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36325d4-79b9-480c-975a-c72a4f60b2c0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:00.351323 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36325d4-79b9-480c-975a-c72a4f60b2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:00.383769 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a36325d4-79b9-480c-975a-c72a4f60b2c0 None None] MFA Rules not processed for user `615cc36e21cc493b823e6cee695ac308`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:00.385056 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a36325d4-79b9-480c-975a-c72a4f60b2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:00.389712 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 792/1584] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:14:00 2022] POST /identity/v3/projects => generated 375 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:00.391124 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f513fb-5005-41e8-8ae7-b1015bbc21e6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:00.470855 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f513fb-5005-41e8-8ae7-b1015bbc21e6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:00.478829 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d73f7c-30b4-4546-a7c6-b1f0b6486147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:00.479926 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d73f7c-30b4-4546-a7c6-b1f0b6486147 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:00.480412 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d73f7c-30b4-4546-a7c6-b1f0b6486147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:00.480889 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d73f7c-30b4-4546-a7c6-b1f0b6486147 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:00.482175 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 793/1585] 10.222.0.45 () {58 vars in 1059 bytes} [Wed Aug 10 17:14:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:00.482435 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d73f7c-30b4-4546-a7c6-b1f0b6486147 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:00.482951 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36325d4-79b9-480c-975a-c72a4f60b2c0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:00.485754 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d73f7c-30b4-4546-a7c6-b1f0b6486147 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:00.494956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36325d4-79b9-480c-975a-c72a4f60b2c0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:00.500866 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5915f16-2844-4bee-99fd-1f30405bd533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:00.501226 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:00.501504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5915f16-2844-4bee-99fd-1f30405bd533 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:00.501792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5915f16-2844-4bee-99fd-1f30405bd533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:00.502088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5915f16-2844-4bee-99fd-1f30405bd533 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:00.502732 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5915f16-2844-4bee-99fd-1f30405bd533 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:00.504747 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5915f16-2844-4bee-99fd-1f30405bd533 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:00.612900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 793/1586] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:14:00 2022] POST /identity/v3/users => generated 289 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:00.615106 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d73f7c-30b4-4546-a7c6-b1f0b6486147 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:00.621525 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 794/1587] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:14:00 2022] POST /identity/v3/users => generated 292 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:00.626082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5915f16-2844-4bee-99fd-1f30405bd533 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:00.633279 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d73f7c-30b4-4546-a7c6-b1f0b6486147 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:00.643491 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:00.644112 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc26bb1-ecb7-401f-8fba-cc68654ef04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:00.644267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5915f16-2844-4bee-99fd-1f30405bd533 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:00.644382 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc26bb1-ecb7-401f-8fba-cc68654ef04f admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:00.644382 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc26bb1-ecb7-401f-8fba-cc68654ef04f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:00.644825 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc26bb1-ecb7-401f-8fba-cc68654ef04f admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:00.645931 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc26bb1-ecb7-401f-8fba-cc68654ef04f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:00.648889 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc26bb1-ecb7-401f-8fba-cc68654ef04f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:00.650825 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ecccf-9adb-4b11-a744-488bd2cb7071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:00.652681 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ecccf-9adb-4b11-a744-488bd2cb7071 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:00.653168 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ecccf-9adb-4b11-a744-488bd2cb7071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:00.653674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ecccf-9adb-4b11-a744-488bd2cb7071 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:00.654813 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ecccf-9adb-4b11-a744-488bd2cb7071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:00.656573 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ecccf-9adb-4b11-a744-488bd2cb7071 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:00.666056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 794/1588] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:00.668595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc26bb1-ecb7-401f-8fba-cc68654ef04f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:00.675141 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 795/1589] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:00.676626 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ecccf-9adb-4b11-a744-488bd2cb7071 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:00.687212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc26bb1-ecb7-401f-8fba-cc68654ef04f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:00.694394 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdae661-0724-4956-a9ac-1e423f85b15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:00.694679 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ecccf-9adb-4b11-a744-488bd2cb7071 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:00.695443 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdae661-0724-4956-a9ac-1e423f85b15a admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:00.695860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdae661-0724-4956-a9ac-1e423f85b15a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:00.696315 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdae661-0724-4956-a9ac-1e423f85b15a admin admin] PATH_INFO: `/v3/projects/060a1146196f42a494e830743546193e/users/651762b4c7b24909be6fae85b2a16858/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:00.702888 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2d08e-01d4-4a16-983f-bd8b296cd6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:00.710956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2d08e-01d4-4a16-983f-bd8b296cd6de admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:00.710956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2d08e-01d4-4a16-983f-bd8b296cd6de admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:00.710956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2d08e-01d4-4a16-983f-bd8b296cd6de admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:00.710956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2d08e-01d4-4a16-983f-bd8b296cd6de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:00.710956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2d08e-01d4-4a16-983f-bd8b296cd6de admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:00.718643 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdae661-0724-4956-a9ac-1e423f85b15a admin admin] RBAC: Authorizing `identity:create_grant(project_id=060a1146196f42a494e830743546193e, user_id=651762b4c7b24909be6fae85b2a16858, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:00.723076 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdae661-0724-4956-a9ac-1e423f85b15a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:00.809854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 796/1590] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:14:00 2022] POST /identity/v3/users => generated 294 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:00.811360 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2d08e-01d4-4a16-983f-bd8b296cd6de admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:00.814076 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 795/1591] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:14:00 2022] PUT /identity/v3/projects/060a1146196f42a494e830743546193e/users/651762b4c7b24909be6fae85b2a16858/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:00.815604 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdae661-0724-4956-a9ac-1e423f85b15a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:00.825912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2d08e-01d4-4a16-983f-bd8b296cd6de admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:00.907834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1e109-0315-4504-ac82-61c613cd32b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:00.908673 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1e109-0315-4504-ac82-61c613cd32b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:00.909056 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1e109-0315-4504-ac82-61c613cd32b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:00.909432 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1e109-0315-4504-ac82-61c613cd32b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:00.910266 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e1e109-0315-4504-ac82-61c613cd32b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:00.912607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e1e109-0315-4504-ac82-61c613cd32b6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:00.927862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdae661-0724-4956-a9ac-1e423f85b15a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:00.934117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d875bfb-4d13-457b-81c0-40d8cc4975e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:00.934781 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d875bfb-4d13-457b-81c0-40d8cc4975e1 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:00.935062 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d875bfb-4d13-457b-81c0-40d8cc4975e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:00.935339 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d875bfb-4d13-457b-81c0-40d8cc4975e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:00.997588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d875bfb-4d13-457b-81c0-40d8cc4975e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:00.999257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d875bfb-4d13-457b-81c0-40d8cc4975e1 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:01.009431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 797/1592] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:14:00 2022] POST /identity/v3/users => generated 288 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:01.024372 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 796/1593] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:01.024372 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d875bfb-4d13-457b-81c0-40d8cc4975e1 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:01.024787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1e109-0315-4504-ac82-61c613cd32b6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:01.024886 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d875bfb-4d13-457b-81c0-40d8cc4975e1 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:01.028844 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1e109-0315-4504-ac82-61c613cd32b6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:01.029293 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de5007-69e1-45f3-ac13-e156705fd309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:01.029923 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de5007-69e1-45f3-ac13-e156705fd309 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:01.030439 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de5007-69e1-45f3-ac13-e156705fd309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:01.030654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de5007-69e1-45f3-ac13-e156705fd309 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:01.031254 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de5007-69e1-45f3-ac13-e156705fd309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:01.032884 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de5007-69e1-45f3-ac13-e156705fd309 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:01.035281 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8584de-2297-4683-84cd-1cef21b54b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:01.036260 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8584de-2297-4683-84cd-1cef21b54b01 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:01.036740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8584de-2297-4683-84cd-1cef21b54b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:01.037134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8584de-2297-4683-84cd-1cef21b54b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:01.037983 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8584de-2297-4683-84cd-1cef21b54b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:01.040270 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8584de-2297-4683-84cd-1cef21b54b01 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:01.045521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 797/1594] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:01.046157 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de5007-69e1-45f3-ac13-e156705fd309 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:01.060961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 798/1595] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:01.061203 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de5007-69e1-45f3-ac13-e156705fd309 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:01.061381 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8584de-2297-4683-84cd-1cef21b54b01 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:01.065993 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbeb08-cd5c-4e0b-9706-0929812f2362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:01.066817 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbeb08-cd5c-4e0b-9706-0929812f2362 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:01.067164 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbeb08-cd5c-4e0b-9706-0929812f2362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:01.067509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbeb08-cd5c-4e0b-9706-0929812f2362 admin admin] PATH_INFO: `/v3/projects/2eb3ea42180f4a0ca5ca3c90962fb301/users/7d2a65984c4f42fbb18977c7345c42ce/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:01.076277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8584de-2297-4683-84cd-1cef21b54b01 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:01.083044 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8e607-8d0a-4c5a-b272-2442b5f38f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:01.084205 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8e607-8d0a-4c5a-b272-2442b5f38f5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:01.084766 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8e607-8d0a-4c5a-b272-2442b5f38f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:01.085311 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8e607-8d0a-4c5a-b272-2442b5f38f5e admin admin] PATH_INFO: `/v3/projects/8a7cd3e7d80845e89b398f0707097ffd/users/06cb7a8f70ab409b86c669620df4a651/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:01.091928 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dbeb08-cd5c-4e0b-9706-0929812f2362 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eb3ea42180f4a0ca5ca3c90962fb301, user_id=7d2a65984c4f42fbb18977c7345c42ce, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:01.095873 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dbeb08-cd5c-4e0b-9706-0929812f2362 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:01.105525 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb8e607-8d0a-4c5a-b272-2442b5f38f5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a7cd3e7d80845e89b398f0707097ffd, user_id=06cb7a8f70ab409b86c669620df4a651, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:01.108713 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb8e607-8d0a-4c5a-b272-2442b5f38f5e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:01.178685 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 798/1596] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:14:01 2022] PUT /identity/v3/projects/2eb3ea42180f4a0ca5ca3c90962fb301/users/7d2a65984c4f42fbb18977c7345c42ce/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:01.179230 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbeb08-cd5c-4e0b-9706-0929812f2362 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:01.209425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:01.264980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 799/1597] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:14:01 2022] PUT /identity/v3/projects/8a7cd3e7d80845e89b398f0707097ffd/users/06cb7a8f70ab409b86c669620df4a651/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:01.267585 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b895fad-472f-4bb7-9912-2105d7e69edb None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:01.276652 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b895fad-472f-4bb7-9912-2105d7e69edb None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:01.276652 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b895fad-472f-4bb7-9912-2105d7e69edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:01.295257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbeb08-cd5c-4e0b-9706-0929812f2362 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:01.317966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1b895fad-472f-4bb7-9912-2105d7e69edb None None] MFA Rules not processed for user `651762b4c7b24909be6fae85b2a16858`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:01.320063 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1b895fad-472f-4bb7-9912-2105d7e69edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:01.399488 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a1591-8ef1-4875-873a-3d6e81f7f56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:01.400566 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a1591-8ef1-4875-873a-3d6e81f7f56c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:01.401068 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a1591-8ef1-4875-873a-3d6e81f7f56c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:01.401581 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a1591-8ef1-4875-873a-3d6e81f7f56c admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:01.402797 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316a1591-8ef1-4875-873a-3d6e81f7f56c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:01.405859 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316a1591-8ef1-4875-873a-3d6e81f7f56c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:01.419459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 800/1598] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:01.421027 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 799/1599] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:01.421180 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b895fad-472f-4bb7-9912-2105d7e69edb None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:01.422296 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a1591-8ef1-4875-873a-3d6e81f7f56c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:01.436473 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a1591-8ef1-4875-873a-3d6e81f7f56c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:01.439523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b895fad-472f-4bb7-9912-2105d7e69edb None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:01.442005 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce3a5a-cbd4-40db-ac7f-740e7ba4f6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:01.442657 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce3a5a-cbd4-40db-ac7f-740e7ba4f6b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:01.442992 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce3a5a-cbd4-40db-ac7f-740e7ba4f6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:01.443362 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce3a5a-cbd4-40db-ac7f-740e7ba4f6b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:01.444016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bce3a5a-cbd4-40db-ac7f-740e7ba4f6b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:01.445540 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3eaa0-8e65-4bb9-8d44-266f13164802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:01.445894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bce3a5a-cbd4-40db-ac7f-740e7ba4f6b7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:01.446477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3eaa0-8e65-4bb9-8d44-266f13164802 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:01.446850 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3eaa0-8e65-4bb9-8d44-266f13164802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:01.447284 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3eaa0-8e65-4bb9-8d44-266f13164802 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:01.448093 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3eaa0-8e65-4bb9-8d44-266f13164802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:01.451057 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3eaa0-8e65-4bb9-8d44-266f13164802 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:01.460891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 800/1600] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:01.461971 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce3a5a-cbd4-40db-ac7f-740e7ba4f6b7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:01.465987 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 801/1601] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:01.471497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb197d6-ce34-4240-b31a-810f9d998986 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:01.471864 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb197d6-ce34-4240-b31a-810f9d998986 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:01.472283 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb197d6-ce34-4240-b31a-810f9d998986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:01.473477 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce3a5a-cbd4-40db-ac7f-740e7ba4f6b7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:01.480529 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd66891-caf2-4bf8-aaa9-53eb112e5684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:01.481976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd66891-caf2-4bf8-aaa9-53eb112e5684 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:01.482467 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd66891-caf2-4bf8-aaa9-53eb112e5684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:01.483007 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd66891-caf2-4bf8-aaa9-53eb112e5684 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:01.484078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd66891-caf2-4bf8-aaa9-53eb112e5684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:01.486233 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd66891-caf2-4bf8-aaa9-53eb112e5684 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:01.502594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6fb197d6-ce34-4240-b31a-810f9d998986 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:01.504510 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6fb197d6-ce34-4240-b31a-810f9d998986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:01.509226 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 801/1602] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:01.509226 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607bb19-cb8c-4f9f-9498-b3b376568760 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:01.509226 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607bb19-cb8c-4f9f-9498-b3b376568760 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:01.509226 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607bb19-cb8c-4f9f-9498-b3b376568760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:01.523026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 802/1603] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:01.523026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880d410-5087-4404-a6ac-415b3182aa47 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:01.523026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880d410-5087-4404-a6ac-415b3182aa47 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:01.523026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880d410-5087-4404-a6ac-415b3182aa47 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:01.524541 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 803/1604] 10.222.0.45 () {54 vars in 912 bytes} [Wed Aug 10 17:14:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:14:01.525286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880d410-5087-4404-a6ac-415b3182aa47 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:01.535479 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:01.542772 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3607bb19-cb8c-4f9f-9498-b3b376568760 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:01.543424 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880d410-5087-4404-a6ac-415b3182aa47 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:01.544983 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3607bb19-cb8c-4f9f-9498-b3b376568760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:01.548674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ed36c-6c02-4d9c-a50d-975b47ac7863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:01.549400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ed36c-6c02-4d9c-a50d-975b47ac7863 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:01.549792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ed36c-6c02-4d9c-a50d-975b47ac7863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:01.550223 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ed36c-6c02-4d9c-a50d-975b47ac7863 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:01.551013 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ed36c-6c02-4d9c-a50d-975b47ac7863 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:01.553178 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ed36c-6c02-4d9c-a50d-975b47ac7863 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:01.560116 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 802/1605] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:01.567543 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607bb19-cb8c-4f9f-9498-b3b376568760 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:01.582339 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607bb19-cb8c-4f9f-9498-b3b376568760 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:01.588590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7d566-fce2-4f6a-bb10-c9d8364549ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:01.590253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7d566-fce2-4f6a-bb10-c9d8364549ba admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:01.590253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7d566-fce2-4f6a-bb10-c9d8364549ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:01.590500 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7d566-fce2-4f6a-bb10-c9d8364549ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:01.591096 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7d566-fce2-4f6a-bb10-c9d8364549ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:01.592715 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7d566-fce2-4f6a-bb10-c9d8364549ba admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:01.603592 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 803/1606] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:01.608204 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7d566-fce2-4f6a-bb10-c9d8364549ba admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:01.621175 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7d566-fce2-4f6a-bb10-c9d8364549ba admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:01.625970 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72bdac-412e-4456-a4a3-6c70e37cf866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:01.626736 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72bdac-412e-4456-a4a3-6c70e37cf866 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:01.627023 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72bdac-412e-4456-a4a3-6c70e37cf866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:01.627302 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72bdac-412e-4456-a4a3-6c70e37cf866 admin admin] PATH_INFO: `/v3/projects/7e92e1e1c27241798e336e547580be5e/users/2f74a20789dc4ef697861146252bacba/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:01.646532 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72bdac-412e-4456-a4a3-6c70e37cf866 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e92e1e1c27241798e336e547580be5e, user_id=2f74a20789dc4ef697861146252bacba, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:01.653794 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72bdac-412e-4456-a4a3-6c70e37cf866 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:01.673512 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 804/1607] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:14:01 2022] POST /identity/v3/projects => generated 377 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:01.674807 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ed36c-6c02-4d9c-a50d-975b47ac7863 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:01.746128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ed36c-6c02-4d9c-a50d-975b47ac7863 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:01.752015 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfc0d5-16cc-4c11-82c4-740850c86aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:01.752617 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfc0d5-16cc-4c11-82c4-740850c86aac admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:01.752885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfc0d5-16cc-4c11-82c4-740850c86aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:01.753160 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfc0d5-16cc-4c11-82c4-740850c86aac admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:01.753814 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bfc0d5-16cc-4c11-82c4-740850c86aac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:01.755491 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bfc0d5-16cc-4c11-82c4-740850c86aac admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:01.770722 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 804/1608] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:14:01 2022] PUT /identity/v3/projects/7e92e1e1c27241798e336e547580be5e/users/2f74a20789dc4ef697861146252bacba/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:01.774145 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 805/1609] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:01.774407 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72bdac-412e-4456-a4a3-6c70e37cf866 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:01.774594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f75ca-5a90-438b-86a1-4d8081062430 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:01.774594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f75ca-5a90-438b-86a1-4d8081062430 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:01.774594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f75ca-5a90-438b-86a1-4d8081062430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:01.810571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-095f75ca-5a90-438b-86a1-4d8081062430 None None] MFA Rules not processed for user `615cc36e21cc493b823e6cee695ac308`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:01.812526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-095f75ca-5a90-438b-86a1-4d8081062430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:01.842029 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:01.846302 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72bdac-412e-4456-a4a3-6c70e37cf866 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:01.851458 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799ef32-8281-41ae-a07e-53ef162263f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:01.852146 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799ef32-8281-41ae-a07e-53ef162263f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:01.852419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799ef32-8281-41ae-a07e-53ef162263f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:01.852694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799ef32-8281-41ae-a07e-53ef162263f3 admin admin] PATH_INFO: `/v3/projects/cfac2c11f1b24456a5a9efaa9112e281/users/48feb2fc02f04e739c5e1bf7d68baa16/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:01.869317 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8799ef32-8281-41ae-a07e-53ef162263f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfac2c11f1b24456a5a9efaa9112e281, user_id=48feb2fc02f04e739c5e1bf7d68baa16, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:01.872824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8799ef32-8281-41ae-a07e-53ef162263f3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:01.931491 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 806/1610] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 10 17:14:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:01.934088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f75ca-5a90-438b-86a1-4d8081062430 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:01.952190 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 805/1611] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:14:01 2022] PUT /identity/v3/projects/cfac2c11f1b24456a5a9efaa9112e281/users/48feb2fc02f04e739c5e1bf7d68baa16/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:01.953078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799ef32-8281-41ae-a07e-53ef162263f3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:02.022296 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799ef32-8281-41ae-a07e-53ef162263f3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:02.027219 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee7583-1bd9-42cf-be1a-5c3a371743d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:02.027885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee7583-1bd9-42cf-be1a-5c3a371743d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:02.028079 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f75ca-5a90-438b-86a1-4d8081062430 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:02.028235 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee7583-1bd9-42cf-be1a-5c3a371743d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:02.028398 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee7583-1bd9-42cf-be1a-5c3a371743d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:02.032004 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e19c0-278f-45be-9ffb-be7ffebd0171 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:02.032656 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e19c0-278f-45be-9ffb-be7ffebd0171 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:02.032880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e19c0-278f-45be-9ffb-be7ffebd0171 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:02.034405 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e19c0-278f-45be-9ffb-be7ffebd0171 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:02.067282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:02.113700 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee7583-1bd9-42cf-be1a-5c3a371743d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:02.116064 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee7583-1bd9-42cf-be1a-5c3a371743d5 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:02.128848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e19c0-278f-45be-9ffb-be7ffebd0171 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:02.129246 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 806/1612] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:02.130574 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee7583-1bd9-42cf-be1a-5c3a371743d5 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:02.131637 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e19c0-278f-45be-9ffb-be7ffebd0171 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:02.147886 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee7583-1bd9-42cf-be1a-5c3a371743d5 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:02.149993 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 807/1613] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:02.152547 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e19c0-278f-45be-9ffb-be7ffebd0171 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:02.159577 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cc7bc-eba6-4e10-bcd8-52ddadf67110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:02.162249 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cc7bc-eba6-4e10-bcd8-52ddadf67110 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:02.162249 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cc7bc-eba6-4e10-bcd8-52ddadf67110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:02.162249 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cc7bc-eba6-4e10-bcd8-52ddadf67110 admin admin] PATH_INFO: `/v3/projects/ed45fd312c58480f993616d451abcab7/users/5bf9229cd5d6429ea14249f8976e83cc/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:02.170608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e19c0-278f-45be-9ffb-be7ffebd0171 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:02.179414 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792bc2b1-f912-4129-918b-0e118a61ac87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:02.180264 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792bc2b1-f912-4129-918b-0e118a61ac87 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:02.180700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792bc2b1-f912-4129-918b-0e118a61ac87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:02.181086 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792bc2b1-f912-4129-918b-0e118a61ac87 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:02.181983 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792bc2b1-f912-4129-918b-0e118a61ac87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:02.186252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792bc2b1-f912-4129-918b-0e118a61ac87 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:02.192910 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6cc7bc-eba6-4e10-bcd8-52ddadf67110 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed45fd312c58480f993616d451abcab7, user_id=5bf9229cd5d6429ea14249f8976e83cc, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:02.197630 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6cc7bc-eba6-4e10-bcd8-52ddadf67110 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:02.293497 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 807/1614] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:14:02 2022] PUT /identity/v3/projects/ed45fd312c58480f993616d451abcab7/users/5bf9229cd5d6429ea14249f8976e83cc/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:02.298791 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 808/1615] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:14:02 2022] POST /identity/v3/users => generated 295 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:02.300330 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cc7bc-eba6-4e10-bcd8-52ddadf67110 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:02.300685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec617051-238f-4c4d-8924-1ff3bc857115 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:02.301108 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec617051-238f-4c4d-8924-1ff3bc857115 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:02.301513 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec617051-238f-4c4d-8924-1ff3bc857115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:02.330767 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ec617051-238f-4c4d-8924-1ff3bc857115 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:02.332329 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ec617051-238f-4c4d-8924-1ff3bc857115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:02.431404 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cc7bc-eba6-4e10-bcd8-52ddadf67110 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:02.437293 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928bd46-f365-417c-a20f-facec38a50dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:02.437982 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928bd46-f365-417c-a20f-facec38a50dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:02.438268 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928bd46-f365-417c-a20f-facec38a50dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:02.438565 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928bd46-f365-417c-a20f-facec38a50dd admin admin] PATH_INFO: `/v3/projects/eaff929dc8a54995bc2eb72576e01495/users/de21b7ddd7f543fe895e94ac57e11f81/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:02.447353 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 809/1616] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:02.449180 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec617051-238f-4c4d-8924-1ff3bc857115 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:02.460117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:02.466329 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d928bd46-f365-417c-a20f-facec38a50dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaff929dc8a54995bc2eb72576e01495, user_id=de21b7ddd7f543fe895e94ac57e11f81, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:02.469328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d928bd46-f365-417c-a20f-facec38a50dd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:02.549250 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec617051-238f-4c4d-8924-1ff3bc857115 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:02.553682 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46eb64-bf77-4b71-beb7-4533da22fdc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:02.557261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46eb64-bf77-4b71-beb7-4533da22fdc7 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:02.557261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46eb64-bf77-4b71-beb7-4533da22fdc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:02.557261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46eb64-bf77-4b71-beb7-4533da22fdc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:02.595977 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 808/1617] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:14:02 2022] PUT /identity/v3/projects/eaff929dc8a54995bc2eb72576e01495/users/de21b7ddd7f543fe895e94ac57e11f81/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:02.599390 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992c33d-6114-48c8-b932-2a8699e5dd5e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:02.599881 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992c33d-6114-48c8-b932-2a8699e5dd5e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:02.600363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992c33d-6114-48c8-b932-2a8699e5dd5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:02.641864 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f992c33d-6114-48c8-b932-2a8699e5dd5e None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:02.643690 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f992c33d-6114-48c8-b932-2a8699e5dd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:02.655747 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:02.676693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46eb64-bf77-4b71-beb7-4533da22fdc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:02.679819 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46eb64-bf77-4b71-beb7-4533da22fdc7 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:02.692078 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 810/1618] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:02.693058 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46eb64-bf77-4b71-beb7-4533da22fdc7 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:02.704839 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46eb64-bf77-4b71-beb7-4533da22fdc7 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:02.758868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 809/1619] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:02.760069 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32241d60-8ccc-426c-97df-81154b011047 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:02.760352 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32241d60-8ccc-426c-97df-81154b011047 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:02.760775 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32241d60-8ccc-426c-97df-81154b011047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:02.790020 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c2a1c8-5469-4be4-a000-c759573916a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:02.791251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2a1c8-5469-4be4-a000-c759573916a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:02.791816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2a1c8-5469-4be4-a000-c759573916a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:02.792350 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2a1c8-5469-4be4-a000-c759573916a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:02.793628 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c2a1c8-5469-4be4-a000-c759573916a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:02.797408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c2a1c8-5469-4be4-a000-c759573916a9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:02.802322 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-32241d60-8ccc-426c-97df-81154b011047 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:02.804894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-32241d60-8ccc-426c-97df-81154b011047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:02.818232 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 810/1620] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:02.819423 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32241d60-8ccc-426c-97df-81154b011047 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:02.821835 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-32241d60-8ccc-426c-97df-81154b011047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:02.855070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 811/1621] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:02.857328 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc3e48-dcaf-4fd4-ba48-a1f445999b73 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:02.857817 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc3e48-dcaf-4fd4-ba48-a1f445999b73 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:02.858441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc3e48-dcaf-4fd4-ba48-a1f445999b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:02.929302 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32241d60-8ccc-426c-97df-81154b011047 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:02.936540 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a186a9-90fa-4166-8045-a5d1a8381a3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:02.937132 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-62dc3e48-dcaf-4fd4-ba48-a1f445999b73 None None] MFA Rules not processed for user `de21b7ddd7f543fe895e94ac57e11f81`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:02.937810 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a186a9-90fa-4166-8045-a5d1a8381a3b service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:02.938304 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a186a9-90fa-4166-8045-a5d1a8381a3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:02.939026 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a186a9-90fa-4166-8045-a5d1a8381a3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:02.940614 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-62dc3e48-dcaf-4fd4-ba48-a1f445999b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:02.956058 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a186a9-90fa-4166-8045-a5d1a8381a3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:02.958011 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a186a9-90fa-4166-8045-a5d1a8381a3b service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:02.976050 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 811/1622] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:02.976921 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a186a9-90fa-4166-8045-a5d1a8381a3b service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:02.992281 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a186a9-90fa-4166-8045-a5d1a8381a3b service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:02.997059 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf74411-8e43-4b6d-9111-cc6df42edab6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:02.997804 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf74411-8e43-4b6d-9111-cc6df42edab6 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:02.998206 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf74411-8e43-4b6d-9111-cc6df42edab6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:02.998428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf74411-8e43-4b6d-9111-cc6df42edab6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:03.013564 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf74411-8e43-4b6d-9111-cc6df42edab6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:03.015680 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf74411-8e43-4b6d-9111-cc6df42edab6 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:03.035961 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 812/1623] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:03.035961 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf74411-8e43-4b6d-9111-cc6df42edab6 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:03.045700 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf74411-8e43-4b6d-9111-cc6df42edab6 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:03.051118 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ea20f-95c1-4a77-90c5-bac49e27b089 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:03.051763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ea20f-95c1-4a77-90c5-bac49e27b089 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:03.052077 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ea20f-95c1-4a77-90c5-bac49e27b089 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:03.052430 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ea20f-95c1-4a77-90c5-bac49e27b089 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:03.068573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ea20f-95c1-4a77-90c5-bac49e27b089 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:03.070419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ea20f-95c1-4a77-90c5-bac49e27b089 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:03.084293 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 813/1624] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:03.085363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ea20f-95c1-4a77-90c5-bac49e27b089 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:03.098973 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ea20f-95c1-4a77-90c5-bac49e27b089 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:03.106410 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4471e-c4cf-4d54-b039-89aff59647b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:03.107188 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4471e-c4cf-4d54-b039-89aff59647b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:03.107468 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4471e-c4cf-4d54-b039-89aff59647b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:03.110067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4471e-c4cf-4d54-b039-89aff59647b1 admin admin] PATH_INFO: `/v3/projects/d578d2c4c6da4e00a386db0eb012b071/users/15cb06cc9f8745ec8535ea7ed76a2d03/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:03.128106 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e4471e-c4cf-4d54-b039-89aff59647b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d578d2c4c6da4e00a386db0eb012b071, user_id=15cb06cc9f8745ec8535ea7ed76a2d03, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:03.131878 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 812/1625] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:03.132190 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e4471e-c4cf-4d54-b039-89aff59647b1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:03.157508 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c950ee7-319d-47dd-9294-eef3c0771f21 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:03.157774 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c950ee7-319d-47dd-9294-eef3c0771f21 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:03.158062 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c950ee7-319d-47dd-9294-eef3c0771f21 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:03.159932 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 813/1626] 10.222.0.45 () {54 vars in 962 bytes} [Wed Aug 10 17:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 17:14:03.167929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a099c2-018c-4869-8f85-e0da326e6d9d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:03.169039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a099c2-018c-4869-8f85-e0da326e6d9d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:03.169039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a099c2-018c-4869-8f85-e0da326e6d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:03.193521 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-64a099c2-018c-4869-8f85-e0da326e6d9d None None] MFA Rules not processed for user `615cc36e21cc493b823e6cee695ac308`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:03.194914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-64a099c2-018c-4869-8f85-e0da326e6d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:03.282434 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:03.307122 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 814/1627] 10.222.0.45 () {58 vars in 1059 bytes} [Wed Aug 10 17:14:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:03.323155 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a099c2-018c-4869-8f85-e0da326e6d9d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:03.352856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a099c2-018c-4869-8f85-e0da326e6d9d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:03.367725 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 814/1628] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:14:03 2022] PUT /identity/v3/projects/d578d2c4c6da4e00a386db0eb012b071/users/15cb06cc9f8745ec8535ea7ed76a2d03/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:03.374209 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4471e-c4cf-4d54-b039-89aff59647b1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:03.467099 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:03.548335 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4471e-c4cf-4d54-b039-89aff59647b1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:03.562139 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e454b0-ce12-4949-854c-9d0128c5fb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:03.563513 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e454b0-ce12-4949-854c-9d0128c5fb93 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:03.564009 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e454b0-ce12-4949-854c-9d0128c5fb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:03.564626 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e454b0-ce12-4949-854c-9d0128c5fb93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:03.569490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e454b0-ce12-4949-854c-9d0128c5fb93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:03.575225 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e454b0-ce12-4949-854c-9d0128c5fb93 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:03.592249 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f731d4-b907-4eda-a5a3-1294b6795725 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:03.595556 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f731d4-b907-4eda-a5a3-1294b6795725 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:03.597569 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f731d4-b907-4eda-a5a3-1294b6795725 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:03.599881 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f731d4-b907-4eda-a5a3-1294b6795725 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:03.634936 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 815/1629] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:03.642920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e454b0-ce12-4949-854c-9d0128c5fb93 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:03.669007 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e454b0-ce12-4949-854c-9d0128c5fb93 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:03.681510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00418e4a-0f0d-4522-bb78-70ad62e835ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:03.683122 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00418e4a-0f0d-4522-bb78-70ad62e835ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:03.683959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00418e4a-0f0d-4522-bb78-70ad62e835ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:03.684657 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00418e4a-0f0d-4522-bb78-70ad62e835ce admin admin] PATH_INFO: `/v3/projects/d578d2c4c6da4e00a386db0eb012b071/users/15cb06cc9f8745ec8535ea7ed76a2d03/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:03.689039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00418e4a-0f0d-4522-bb78-70ad62e835ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=d578d2c4c6da4e00a386db0eb012b071, user_id=15cb06cc9f8745ec8535ea7ed76a2d03, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:03.696929 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00418e4a-0f0d-4522-bb78-70ad62e835ce admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:03.720361 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 816/1630] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:14:03 2022] PUT /identity/v3/projects/d578d2c4c6da4e00a386db0eb012b071/users/15cb06cc9f8745ec8535ea7ed76a2d03/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:14:03.745260 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c770a-2a5a-4d0f-b70a-e211b8c34216 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:03.745643 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c770a-2a5a-4d0f-b70a-e211b8c34216 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:03.746128 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c770a-2a5a-4d0f-b70a-e211b8c34216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:03.822644 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f731d4-b907-4eda-a5a3-1294b6795725 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:03.824586 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f731d4-b907-4eda-a5a3-1294b6795725 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:03.849857 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-933c770a-2a5a-4d0f-b70a-e211b8c34216 None None] MFA Rules not processed for user `15cb06cc9f8745ec8535ea7ed76a2d03`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:03.851928 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 815/1631] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:14:03.853240 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-933c770a-2a5a-4d0f-b70a-e211b8c34216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:04.048653 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 817/1632] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:04.071979 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f731d4-b907-4eda-a5a3-1294b6795725 service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:04.245642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f731d4-b907-4eda-a5a3-1294b6795725 service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:04.306938 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d62cb-507d-4a7d-b08c-b0fa73dca7f9 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:04.307689 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d62cb-507d-4a7d-b08c-b0fa73dca7f9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:04.311722 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d62cb-507d-4a7d-b08c-b0fa73dca7f9 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:04.311722 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 818/1633] 10.222.0.45 () {54 vars in 912 bytes} [Wed Aug 10 17:14:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:14:04.331412 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd365a-9d79-4182-b43b-725f0f84785c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:04.336159 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd365a-9d79-4182-b43b-725f0f84785c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:04.337272 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd365a-9d79-4182-b43b-725f0f84785c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:04.383999 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-cbcd365a-9d79-4182-b43b-725f0f84785c None None] MFA Rules not processed for user `615cc36e21cc493b823e6cee695ac308`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:04.387654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cbcd365a-9d79-4182-b43b-725f0f84785c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:04.398563 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f009e-f192-42fe-9805-eda033a0edb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:04.399194 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f009e-f192-42fe-9805-eda033a0edb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:04.399525 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f009e-f192-42fe-9805-eda033a0edb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:04.399781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f009e-f192-42fe-9805-eda033a0edb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:04.418776 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:04.422853 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f009e-f192-42fe-9805-eda033a0edb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:04.424305 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f009e-f192-42fe-9805-eda033a0edb5 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:04.451927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 816/1634] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:04.647502 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 819/1635] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 10 17:14:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:04.692989 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f009e-f192-42fe-9805-eda033a0edb5 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:04.715607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f009e-f192-42fe-9805-eda033a0edb5 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:04.724444 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf01181-0296-4970-b855-54c71da8c34a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:04.725220 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf01181-0296-4970-b855-54c71da8c34a service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:04.725878 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf01181-0296-4970-b855-54c71da8c34a service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:04.726383 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf01181-0296-4970-b855-54c71da8c34a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:04.759651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf01181-0296-4970-b855-54c71da8c34a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:04.759651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf01181-0296-4970-b855-54c71da8c34a service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:04.806086 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 817/1636] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:04.998408 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd365a-9d79-4182-b43b-725f0f84785c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:05.021157 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd365a-9d79-4182-b43b-725f0f84785c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:05.028813 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b766873-1f74-4f49-8e0f-f0c5f5e9c9fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:05.030302 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b766873-1f74-4f49-8e0f-f0c5f5e9c9fd service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:05.032209 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b766873-1f74-4f49-8e0f-f0c5f5e9c9fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:05.033979 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b766873-1f74-4f49-8e0f-f0c5f5e9c9fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:05.390765 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:05.403091 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf01181-0296-4970-b855-54c71da8c34a service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:05.409701 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b766873-1f74-4f49-8e0f-f0c5f5e9c9fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:05.409701 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b766873-1f74-4f49-8e0f-f0c5f5e9c9fd service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:05.409701 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 820/1637] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:05.467776 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf01181-0296-4970-b855-54c71da8c34a service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:05.483104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2437bc-f2d4-4e48-b461-fce1bf7de90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:05.490269 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2437bc-f2d4-4e48-b461-fce1bf7de90b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:05.492006 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2437bc-f2d4-4e48-b461-fce1bf7de90b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:05.492873 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2437bc-f2d4-4e48-b461-fce1bf7de90b admin admin] PATH_INFO: `/v3/users/de21b7ddd7f543fe895e94ac57e11f81` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:05.504131 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2437bc-f2d4-4e48-b461-fce1bf7de90b admin admin] RBAC: Authorizing `identity:delete_user(user_id=de21b7ddd7f543fe895e94ac57e11f81)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:05.511622 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2437bc-f2d4-4e48-b461-fce1bf7de90b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:05.832432 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ef2437bc-f2d4-4e48-b461-fce1bf7de90b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de21b7ddd7f543fe895e94ac57e11f81'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:05.937258 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ef2437bc-f2d4-4e48-b461-fce1bf7de90b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de21b7ddd7f543fe895e94ac57e11f81'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:05.972766 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ef2437bc-f2d4-4e48-b461-fce1bf7de90b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de21b7ddd7f543fe895e94ac57e11f81'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:05.974143 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ef2437bc-f2d4-4e48-b461-fce1bf7de90b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de21b7ddd7f543fe895e94ac57e11f81'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:05.975221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ef2437bc-f2d4-4e48-b461-fce1bf7de90b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de21b7ddd7f543fe895e94ac57e11f81'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:06.020797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 818/1638] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:14:05 2022] DELETE /identity/v3/users/de21b7ddd7f543fe895e94ac57e11f81 => generated 0 bytes in 774 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:06.030831 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b06ae-3151-450f-9888-0b8f7e0e9473 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:06.031627 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b06ae-3151-450f-9888-0b8f7e0e9473 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:06.031960 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b06ae-3151-450f-9888-0b8f7e0e9473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:06.100311 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-252b06ae-3151-450f-9888-0b8f7e0e9473 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:06.103466 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-252b06ae-3151-450f-9888-0b8f7e0e9473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:06.498663 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 821/1639] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:14:06.524729 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2437bc-f2d4-4e48-b461-fce1bf7de90b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:06.529189 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ef2437bc-f2d4-4e48-b461-fce1bf7de90b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:06.756669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2437bc-f2d4-4e48-b461-fce1bf7de90b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:06.771964 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6335a-50da-409f-960c-e98c14e6f610 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:06.773025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6335a-50da-409f-960c-e98c14e6f610 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:06.773501 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6335a-50da-409f-960c-e98c14e6f610 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:06.773945 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6335a-50da-409f-960c-e98c14e6f610 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:06.809154 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c6335a-50da-409f-960c-e98c14e6f610 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:06.812995 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c6335a-50da-409f-960c-e98c14e6f610 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:06.836374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 819/1640] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:14:08.237645 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b06ae-3151-450f-9888-0b8f7e0e9473 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:08.239277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-252b06ae-3151-450f-9888-0b8f7e0e9473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:08.264033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b06ae-3151-450f-9888-0b8f7e0e9473 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:08.273835 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31561738-f678-418d-9683-3b54a172d299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:08.274820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31561738-f678-418d-9683-3b54a172d299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:08.275240 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31561738-f678-418d-9683-3b54a172d299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:08.275803 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31561738-f678-418d-9683-3b54a172d299 admin admin] PATH_INFO: `/v3/users/e08cf8c5e0a0497d80afae2f02a24972` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:08.287322 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31561738-f678-418d-9683-3b54a172d299 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e08cf8c5e0a0497d80afae2f02a24972)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:08.288813 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-31561738-f678-418d-9683-3b54a172d299 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 17:14:08.288813 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Aug 10 17:14:08.288813 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: Aug 10 17:14:08.290393 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31561738-f678-418d-9683-3b54a172d299 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:08.440517 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6335a-50da-409f-960c-e98c14e6f610 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:08.442630 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d9c6335a-50da-409f-960c-e98c14e6f610 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:08.460643 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:08.473070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6335a-50da-409f-960c-e98c14e6f610 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:08.488929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519474df-35c3-4c2c-8027-e338e4ac31a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:08.488929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519474df-35c3-4c2c-8027-e338e4ac31a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:08.488929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519474df-35c3-4c2c-8027-e338e4ac31a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:08.488929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519474df-35c3-4c2c-8027-e338e4ac31a3 admin admin] PATH_INFO: `/v3/projects/eaff929dc8a54995bc2eb72576e01495` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:08.493440 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519474df-35c3-4c2c-8027-e338e4ac31a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaff929dc8a54995bc2eb72576e01495)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:08.494768 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-519474df-35c3-4c2c-8027-e338e4ac31a3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 17:14:08.494768 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Aug 10 17:14:08.494768 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: Aug 10 17:14:08.498253 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519474df-35c3-4c2c-8027-e338e4ac31a3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:08.573655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-31561738-f678-418d-9683-3b54a172d299 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e08cf8c5e0a0497d80afae2f02a24972'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:08.696112 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-31561738-f678-418d-9683-3b54a172d299 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e08cf8c5e0a0497d80afae2f02a24972'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:08.738072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-31561738-f678-418d-9683-3b54a172d299 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e08cf8c5e0a0497d80afae2f02a24972'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:08.839448 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-519474df-35c3-4c2c-8027-e338e4ac31a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaff929dc8a54995bc2eb72576e01495', 'request_id': 'req-519474df-35c3-4c2c-8027-e338e4ac31a3'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:08.843019 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-31561738-f678-418d-9683-3b54a172d299 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e08cf8c5e0a0497d80afae2f02a24972'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:08.844840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-31561738-f678-418d-9683-3b54a172d299 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e08cf8c5e0a0497d80afae2f02a24972'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:08.878436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 822/1641] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:14:08 2022] DELETE /identity/v3/users/e08cf8c5e0a0497d80afae2f02a24972 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:08.894243 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44100b-347e-4b67-93d8-cc127b1409a0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:08.894836 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44100b-347e-4b67-93d8-cc127b1409a0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:08.895331 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44100b-347e-4b67-93d8-cc127b1409a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:08.983503 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-519474df-35c3-4c2c-8027-e338e4ac31a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaff929dc8a54995bc2eb72576e01495', 'request_id': 'req-519474df-35c3-4c2c-8027-e338e4ac31a3'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:09.016750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4f44100b-347e-4b67-93d8-cc127b1409a0 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:09.030809 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4f44100b-347e-4b67-93d8-cc127b1409a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:09.070317 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-519474df-35c3-4c2c-8027-e338e4ac31a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaff929dc8a54995bc2eb72576e01495', 'request_id': 'req-519474df-35c3-4c2c-8027-e338e4ac31a3'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:09.111787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-519474df-35c3-4c2c-8027-e338e4ac31a3 admin admin] The token cache is being invalidate because project eaff929dc8a54995bc2eb72576e01495 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:14:09.112085 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-519474df-35c3-4c2c-8027-e338e4ac31a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:09.114382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-519474df-35c3-4c2c-8027-e338e4ac31a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:09.117280 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 820/1642] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:14:08 2022] DELETE /identity/v3/projects/eaff929dc8a54995bc2eb72576e01495 => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:14:09.122819 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7da6a7-c89b-4813-9391-e2743248ea25 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:09.123026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7da6a7-c89b-4813-9391-e2743248ea25 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:09.123284 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7da6a7-c89b-4813-9391-e2743248ea25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:09.126081 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:09.170556 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2d7da6a7-c89b-4813-9391-e2743248ea25 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:09.172022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2d7da6a7-c89b-4813-9391-e2743248ea25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:09.173057 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 823/1643] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:09.184694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f44100b-347e-4b67-93d8-cc127b1409a0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:09.186590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4f44100b-347e-4b67-93d8-cc127b1409a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:09.202330 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 821/1644] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:09.210751 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66475d7-5127-4e5b-b51a-e92d31221229 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:09.218549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66475d7-5127-4e5b-b51a-e92d31221229 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:09.218549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66475d7-5127-4e5b-b51a-e92d31221229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:09.268371 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c66475d7-5127-4e5b-b51a-e92d31221229 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:09.272242 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c66475d7-5127-4e5b-b51a-e92d31221229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:09.286610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 822/1645] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:09.288402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66475d7-5127-4e5b-b51a-e92d31221229 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:09.302578 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66475d7-5127-4e5b-b51a-e92d31221229 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:09.310409 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f44100b-347e-4b67-93d8-cc127b1409a0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:09.313713 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2c59d-1c1c-477d-a3cf-4779726eca3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:09.315000 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2c59d-1c1c-477d-a3cf-4779726eca3c admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:09.315416 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2c59d-1c1c-477d-a3cf-4779726eca3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:09.315773 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2c59d-1c1c-477d-a3cf-4779726eca3c admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:09.317395 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2c59d-1c1c-477d-a3cf-4779726eca3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:14:09.317852 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe3554-8e4f-435b-8cbc-9f65bf147061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:09.318402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2c59d-1c1c-477d-a3cf-4779726eca3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:09.319096 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe3554-8e4f-435b-8cbc-9f65bf147061 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:09.319657 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe3554-8e4f-435b-8cbc-9f65bf147061 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:09.320231 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe3554-8e4f-435b-8cbc-9f65bf147061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:09.321225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2c59d-1c1c-477d-a3cf-4779726eca3c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:09.353143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe3554-8e4f-435b-8cbc-9f65bf147061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:09.358287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 823/1646] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:14:09.358500 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe3554-8e4f-435b-8cbc-9f65bf147061 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:09.361746 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2c59d-1c1c-477d-a3cf-4779726eca3c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:09.378844 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2c59d-1c1c-477d-a3cf-4779726eca3c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:09.384602 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 824/1647] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:09.393601 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c79d19-5d18-442f-a7bf-46840ee5c1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:09.395106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c79d19-5d18-442f-a7bf-46840ee5c1cd admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:09.395535 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c79d19-5d18-442f-a7bf-46840ee5c1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:09.396192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c79d19-5d18-442f-a7bf-46840ee5c1cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:09.397443 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c79d19-5d18-442f-a7bf-46840ee5c1cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:09.399535 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c79d19-5d18-442f-a7bf-46840ee5c1cd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:09.577522 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 824/1648] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:14:09 2022] POST /identity/v3/projects => generated 363 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:09.580998 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe3554-8e4f-435b-8cbc-9f65bf147061 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:09.623303 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe3554-8e4f-435b-8cbc-9f65bf147061 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:09.790543 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85094131-8ec4-4587-84bc-8bcb21d1c03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:09.791280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85094131-8ec4-4587-84bc-8bcb21d1c03e admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:09.791631 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85094131-8ec4-4587-84bc-8bcb21d1c03e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:09.792712 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85094131-8ec4-4587-84bc-8bcb21d1c03e admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:09.793302 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85094131-8ec4-4587-84bc-8bcb21d1c03e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:09.799248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85094131-8ec4-4587-84bc-8bcb21d1c03e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:09.914005 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 825/1649] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:14:09 2022] POST /identity/v3/users => generated 282 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:09.919735 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c79d19-5d18-442f-a7bf-46840ee5c1cd admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:09.951607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c79d19-5d18-442f-a7bf-46840ee5c1cd admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:09.982739 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef1137-f943-4950-b7ae-bbfd9a91869e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:09.983711 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef1137-f943-4950-b7ae-bbfd9a91869e admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:09.984300 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef1137-f943-4950-b7ae-bbfd9a91869e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:09.984711 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef1137-f943-4950-b7ae-bbfd9a91869e admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:09.986024 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef1137-f943-4950-b7ae-bbfd9a91869e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:09.988626 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef1137-f943-4950-b7ae-bbfd9a91869e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:10.012381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85094131-8ec4-4587-84bc-8bcb21d1c03e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:10.014069 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 825/1650] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:14:10.028993 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85094131-8ec4-4587-84bc-8bcb21d1c03e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:10.042150 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb3ba94-7e52-4339-a1aa-22025570892a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:10.043298 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3ba94-7e52-4339-a1aa-22025570892a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:10.043730 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3ba94-7e52-4339-a1aa-22025570892a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:10.044305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3ba94-7e52-4339-a1aa-22025570892a admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:10.045137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb3ba94-7e52-4339-a1aa-22025570892a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:10.047081 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb3ba94-7e52-4339-a1aa-22025570892a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:10.079777 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 826/1651] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:10.084492 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef1137-f943-4950-b7ae-bbfd9a91869e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:10.105250 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef1137-f943-4950-b7ae-bbfd9a91869e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:10.125159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91f40c-9cbb-4d9e-b4a9-4b7d8d495f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:10.126930 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91f40c-9cbb-4d9e-b4a9-4b7d8d495f2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:10.127386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91f40c-9cbb-4d9e-b4a9-4b7d8d495f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:10.127800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91f40c-9cbb-4d9e-b4a9-4b7d8d495f2f admin admin] PATH_INFO: `/v3/projects/c79dc0bc769c492180d10025bbea56cc/users/ad47c1f969ae4cf4867e2e3db565e2e9/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:10.201724 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed91f40c-9cbb-4d9e-b4a9-4b7d8d495f2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c79dc0bc769c492180d10025bbea56cc, user_id=ad47c1f969ae4cf4867e2e3db565e2e9, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:10.206498 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed91f40c-9cbb-4d9e-b4a9-4b7d8d495f2f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:10.349619 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 826/1652] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:14:10 2022] PUT /identity/v3/projects/c79dc0bc769c492180d10025bbea56cc/users/ad47c1f969ae4cf4867e2e3db565e2e9/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:10.360555 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e3696-5365-4c86-b028-855a766a9b88 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:10.361466 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e3696-5365-4c86-b028-855a766a9b88 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:10.361888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e3696-5365-4c86-b028-855a766a9b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:10.424740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8d2e3696-5365-4c86-b028-855a766a9b88 None None] MFA Rules not processed for user `ad47c1f969ae4cf4867e2e3db565e2e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:10.426141 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8d2e3696-5365-4c86-b028-855a766a9b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:10.603111 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:10.623204 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 827/1653] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:10.628630 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91f40c-9cbb-4d9e-b4a9-4b7d8d495f2f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:10.875376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91f40c-9cbb-4d9e-b4a9-4b7d8d495f2f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:10.890532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110e794-fb4d-465b-b25a-798fd3c1c8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:10.899988 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110e794-fb4d-465b-b25a-798fd3c1c8b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:10.900801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110e794-fb4d-465b-b25a-798fd3c1c8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:10.901899 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110e794-fb4d-465b-b25a-798fd3c1c8b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:10.905495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110e794-fb4d-465b-b25a-798fd3c1c8b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:10.910543 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110e794-fb4d-465b-b25a-798fd3c1c8b0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:10.971070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:11.316213 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 827/1654] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:14:10 2022] POST /identity/v3/projects => generated 363 bytes in 688 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:11.322599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e3696-5365-4c86-b028-855a766a9b88 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:11.343347 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e3696-5365-4c86-b028-855a766a9b88 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:11.373012 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110e794-fb4d-465b-b25a-798fd3c1c8b0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:11.377510 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c110e794-fb4d-465b-b25a-798fd3c1c8b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:11.504911 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638b64f-f925-4684-b0e6-e4742543f574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:11.505691 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638b64f-f925-4684-b0e6-e4742543f574 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:11.505809 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638b64f-f925-4684-b0e6-e4742543f574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:11.506046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638b64f-f925-4684-b0e6-e4742543f574 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:11.506880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638b64f-f925-4684-b0e6-e4742543f574 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:11.519435 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638b64f-f925-4684-b0e6-e4742543f574 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:11.655694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 828/1655] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:14:11 2022] POST /identity/v3/users => generated 288 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:11.663689 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638b64f-f925-4684-b0e6-e4742543f574 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:11.670936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110e794-fb4d-465b-b25a-798fd3c1c8b0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:11.687082 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638b64f-f925-4684-b0e6-e4742543f574 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:11.692432 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae6d3c-a6d8-43f6-911c-4abe63aab3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:11.696383 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae6d3c-a6d8-43f6-911c-4abe63aab3aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:11.708542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae6d3c-a6d8-43f6-911c-4abe63aab3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:11.708542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae6d3c-a6d8-43f6-911c-4abe63aab3aa admin admin] PATH_INFO: `/v3/projects/651ed58528604503882d8c2fa3313a73` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:11.708542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae6d3c-a6d8-43f6-911c-4abe63aab3aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=651ed58528604503882d8c2fa3313a73)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:11.720888 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae6d3c-a6d8-43f6-911c-4abe63aab3aa admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:11.722660 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bd208-53a6-48e4-9c9e-c94f7981eb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:11.723716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bd208-53a6-48e4-9c9e-c94f7981eb81 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:11.724144 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bd208-53a6-48e4-9c9e-c94f7981eb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:11.724753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bd208-53a6-48e4-9c9e-c94f7981eb81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:11.725586 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34bd208-53a6-48e4-9c9e-c94f7981eb81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:11.733114 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34bd208-53a6-48e4-9c9e-c94f7981eb81 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:11.758535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 829/1656] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:11.770500 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bd208-53a6-48e4-9c9e-c94f7981eb81 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:11.818853 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bd208-53a6-48e4-9c9e-c94f7981eb81 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:11.838216 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e74952-68ff-418e-acf3-278d8ffd2bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:11.841465 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e74952-68ff-418e-acf3-278d8ffd2bb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:11.841465 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e74952-68ff-418e-acf3-278d8ffd2bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:11.845646 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e74952-68ff-418e-acf3-278d8ffd2bb2 admin admin] PATH_INFO: `/v3/projects/14f6d7c272164667b20a3ba7727a0a13/users/e09f48a88bf445e0af921b6515f8d80e/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:11.888551 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e74952-68ff-418e-acf3-278d8ffd2bb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14f6d7c272164667b20a3ba7727a0a13, user_id=e09f48a88bf445e0af921b6515f8d80e, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:11.891327 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e74952-68ff-418e-acf3-278d8ffd2bb2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:11.993078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 830/1657] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:14:11 2022] PUT /identity/v3/projects/14f6d7c272164667b20a3ba7727a0a13/users/e09f48a88bf445e0af921b6515f8d80e/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:11.997778 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e74952-68ff-418e-acf3-278d8ffd2bb2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:12.024888 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-19ae6d3c-a6d8-43f6-911c-4abe63aab3aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '651ed58528604503882d8c2fa3313a73', 'request_id': 'req-19ae6d3c-a6d8-43f6-911c-4abe63aab3aa'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:12.160249 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-19ae6d3c-a6d8-43f6-911c-4abe63aab3aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '651ed58528604503882d8c2fa3313a73', 'request_id': 'req-19ae6d3c-a6d8-43f6-911c-4abe63aab3aa'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:12.176454 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-19ae6d3c-a6d8-43f6-911c-4abe63aab3aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '651ed58528604503882d8c2fa3313a73', 'request_id': 'req-19ae6d3c-a6d8-43f6-911c-4abe63aab3aa'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:12.191863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e74952-68ff-418e-acf3-278d8ffd2bb2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:12.216262 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-19ae6d3c-a6d8-43f6-911c-4abe63aab3aa admin admin] The token cache is being invalidate because project 651ed58528604503882d8c2fa3313a73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:14:12.216557 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-19ae6d3c-a6d8-43f6-911c-4abe63aab3aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:12.218079 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-19ae6d3c-a6d8-43f6-911c-4abe63aab3aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:12.223323 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 828/1658] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:14:11 2022] DELETE /identity/v3/projects/651ed58528604503882d8c2fa3313a73 => generated 0 bytes in 848 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:14:12.225001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e898bc4-a655-4eca-9cdc-6738f9475a7a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:12.225284 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e898bc4-a655-4eca-9cdc-6738f9475a7a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:12.225841 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e898bc4-a655-4eca-9cdc-6738f9475a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:12.285280 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1e898bc4-a655-4eca-9cdc-6738f9475a7a None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:12.286865 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1e898bc4-a655-4eca-9cdc-6738f9475a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:12.324756 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870cc0f9-60a4-4df9-9f0e-5dddc80f8f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:12.325610 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cc0f9-60a4-4df9-9f0e-5dddc80f8f97 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:12.326058 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cc0f9-60a4-4df9-9f0e-5dddc80f8f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:12.326865 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cc0f9-60a4-4df9-9f0e-5dddc80f8f97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:12.328005 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870cc0f9-60a4-4df9-9f0e-5dddc80f8f97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:12.331034 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870cc0f9-60a4-4df9-9f0e-5dddc80f8f97 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:12.333145 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:12.349057 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 831/1659] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:14:12.351943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870cc0f9-60a4-4df9-9f0e-5dddc80f8f97 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:12.354119 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-870cc0f9-60a4-4df9-9f0e-5dddc80f8f97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:12.383557 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870cc0f9-60a4-4df9-9f0e-5dddc80f8f97 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:12.395088 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32cbdd8-e980-4ea8-8ed1-05f91335e406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:12.396022 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cbdd8-e980-4ea8-8ed1-05f91335e406 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:12.396485 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cbdd8-e980-4ea8-8ed1-05f91335e406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:12.397223 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cbdd8-e980-4ea8-8ed1-05f91335e406 admin admin] PATH_INFO: `/v3/projects/14f6d7c272164667b20a3ba7727a0a13/users/e09f48a88bf445e0af921b6515f8d80e/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:12.402184 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32cbdd8-e980-4ea8-8ed1-05f91335e406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14f6d7c272164667b20a3ba7727a0a13, user_id=e09f48a88bf445e0af921b6515f8d80e, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:12.406514 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32cbdd8-e980-4ea8-8ed1-05f91335e406 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:12.425810 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 832/1660] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:14:12 2022] PUT /identity/v3/projects/14f6d7c272164667b20a3ba7727a0a13/users/e09f48a88bf445e0af921b6515f8d80e/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:14:12.441699 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5023c8-804e-4633-97e1-6fcd8b0d4451 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:12.442182 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5023c8-804e-4633-97e1-6fcd8b0d4451 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:12.442605 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5023c8-804e-4633-97e1-6fcd8b0d4451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:12.511605 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 829/1661] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:14:12.523536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e1aed-d8b8-44dd-bf12-c31c009b3fca None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:12.524286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e1aed-d8b8-44dd-bf12-c31c009b3fca None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:12.525003 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e1aed-d8b8-44dd-bf12-c31c009b3fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:12.576147 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4a5023c8-804e-4633-97e1-6fcd8b0d4451 None None] MFA Rules not processed for user `e09f48a88bf445e0af921b6515f8d80e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:12.577491 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4a5023c8-804e-4633-97e1-6fcd8b0d4451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:12.653103 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e04e1aed-d8b8-44dd-bf12-c31c009b3fca None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:12.656947 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e04e1aed-d8b8-44dd-bf12-c31c009b3fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:12.704521 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:12.879439 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 833/1662] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:12 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 439 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:12.894620 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 830/1663] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:12.896031 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5023c8-804e-4633-97e1-6fcd8b0d4451 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:12.906300 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e1aed-d8b8-44dd-bf12-c31c009b3fca None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:12.914057 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e04e1aed-d8b8-44dd-bf12-c31c009b3fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:12.957654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5023c8-804e-4633-97e1-6fcd8b0d4451 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:12.974675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ea1e6-e48e-42e7-b97b-3225ee093012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:12.975326 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ea1e6-e48e-42e7-b97b-3225ee093012 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:12.975599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ea1e6-e48e-42e7-b97b-3225ee093012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:12.975864 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ea1e6-e48e-42e7-b97b-3225ee093012 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:12.976551 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ea1e6-e48e-42e7-b97b-3225ee093012 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:14:12.976849 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ea1e6-e48e-42e7-b97b-3225ee093012 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:12.978619 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ea1e6-e48e-42e7-b97b-3225ee093012 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:13.020668 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 834/1664] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:14:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:14:13.026123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ea1e6-e48e-42e7-b97b-3225ee093012 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:13.058074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ea1e6-e48e-42e7-b97b-3225ee093012 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:13.070179 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23032e2e-7b2e-48bc-b40f-0d1c7bdec896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:13.071952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23032e2e-7b2e-48bc-b40f-0d1c7bdec896 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:13.072625 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23032e2e-7b2e-48bc-b40f-0d1c7bdec896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:13.074173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23032e2e-7b2e-48bc-b40f-0d1c7bdec896 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:13.075819 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23032e2e-7b2e-48bc-b40f-0d1c7bdec896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:13.080914 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23032e2e-7b2e-48bc-b40f-0d1c7bdec896 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:13.119711 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e1aed-d8b8-44dd-bf12-c31c009b3fca None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:13.131250 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade8897-ff77-4a3e-868d-97008eda631b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:13.133662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade8897-ff77-4a3e-868d-97008eda631b service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:13.137199 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade8897-ff77-4a3e-868d-97008eda631b service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:13.138669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade8897-ff77-4a3e-868d-97008eda631b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:13.168741 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade8897-ff77-4a3e-868d-97008eda631b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:13.170503 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade8897-ff77-4a3e-868d-97008eda631b service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:13.183909 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 831/1665] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:12 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:13.216502 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 835/1666] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:14:13 2022] POST /identity/v3/projects => generated 373 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:13.217264 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade8897-ff77-4a3e-868d-97008eda631b service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:13.234578 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade8897-ff77-4a3e-868d-97008eda631b service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:13.379854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafe8f0-ba40-4968-9ab5-046ade2b3f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:13.380789 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafe8f0-ba40-4968-9ab5-046ade2b3f3f admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:13.381168 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafe8f0-ba40-4968-9ab5-046ade2b3f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:13.381590 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafe8f0-ba40-4968-9ab5-046ade2b3f3f admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:13.382437 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cafe8f0-ba40-4968-9ab5-046ade2b3f3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:13.384640 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cafe8f0-ba40-4968-9ab5-046ade2b3f3f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:13.487698 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 832/1667] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:14:13 2022] POST /identity/v3/users => generated 287 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:13.492936 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23032e2e-7b2e-48bc-b40f-0d1c7bdec896 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:13.513246 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23032e2e-7b2e-48bc-b40f-0d1c7bdec896 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:13.538112 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5aab3c-9108-40a5-aaa0-77cb0cdcbb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:13.541105 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5aab3c-9108-40a5-aaa0-77cb0cdcbb3e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:13.541392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5aab3c-9108-40a5-aaa0-77cb0cdcbb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:13.541665 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5aab3c-9108-40a5-aaa0-77cb0cdcbb3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:13.547707 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5aab3c-9108-40a5-aaa0-77cb0cdcbb3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:13.552360 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5aab3c-9108-40a5-aaa0-77cb0cdcbb3e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:13.598848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafe8f0-ba40-4968-9ab5-046ade2b3f3f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:13.612222 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 836/1668] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:13.613740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafe8f0-ba40-4968-9ab5-046ade2b3f3f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:13.633447 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bd1cc-fc70-493a-8e08-63f39f4e8b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:13.634121 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bd1cc-fc70-493a-8e08-63f39f4e8b7a admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:13.634424 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bd1cc-fc70-493a-8e08-63f39f4e8b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:13.638820 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bd1cc-fc70-493a-8e08-63f39f4e8b7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:13.645938 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26bd1cc-fc70-493a-8e08-63f39f4e8b7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:13.649104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26bd1cc-fc70-493a-8e08-63f39f4e8b7a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:13.702215 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5aab3c-9108-40a5-aaa0-77cb0cdcbb3e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:13.708225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 833/1669] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:13.719532 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5aab3c-9108-40a5-aaa0-77cb0cdcbb3e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:13.734956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4967fe8-8f25-4675-81f8-edb6ea5d7e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:13.735654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4967fe8-8f25-4675-81f8-edb6ea5d7e02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:13.735933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4967fe8-8f25-4675-81f8-edb6ea5d7e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:13.736207 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4967fe8-8f25-4675-81f8-edb6ea5d7e02 admin admin] PATH_INFO: `/v3/projects/a0b9c7f452594b2c92af6692ac64fde9/users/e7e4aad5aef24a7f8d163a8b07c51e83/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:13.777733 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4967fe8-8f25-4675-81f8-edb6ea5d7e02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0b9c7f452594b2c92af6692ac64fde9, user_id=e7e4aad5aef24a7f8d163a8b07c51e83, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:13.788364 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4967fe8-8f25-4675-81f8-edb6ea5d7e02 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:13.943218 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 837/1670] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:14:13 2022] PUT /identity/v3/projects/a0b9c7f452594b2c92af6692ac64fde9/users/e7e4aad5aef24a7f8d163a8b07c51e83/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:13.956118 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07783a9-9b3b-488d-8f27-3c67ff58c336 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:13.956398 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07783a9-9b3b-488d-8f27-3c67ff58c336 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:13.956839 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07783a9-9b3b-488d-8f27-3c67ff58c336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:14.012310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f07783a9-9b3b-488d-8f27-3c67ff58c336 None None] MFA Rules not processed for user `e7e4aad5aef24a7f8d163a8b07c51e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:14.014486 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f07783a9-9b3b-488d-8f27-3c67ff58c336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:14.093313 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:14.183529 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 834/1671] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:14.183611 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4967fe8-8f25-4675-81f8-edb6ea5d7e02 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:14.303797 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:14.313438 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4967fe8-8f25-4675-81f8-edb6ea5d7e02 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:14.338163 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1735ca2-ed95-494e-9559-833eeb2c3833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:14.339199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1735ca2-ed95-494e-9559-833eeb2c3833 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:14.339496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1735ca2-ed95-494e-9559-833eeb2c3833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:14.342399 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1735ca2-ed95-494e-9559-833eeb2c3833 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:14.343223 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1735ca2-ed95-494e-9559-833eeb2c3833 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:14.347331 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1735ca2-ed95-494e-9559-833eeb2c3833 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:14.513882 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 838/1672] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:14:14 2022] POST /identity/v3/projects => generated 373 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:14.518700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07783a9-9b3b-488d-8f27-3c67ff58c336 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:14.530791 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07783a9-9b3b-488d-8f27-3c67ff58c336 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:14.742779 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402961ef-b1ce-4d0e-9169-18517d5f9285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:14.746312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402961ef-b1ce-4d0e-9169-18517d5f9285 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:14.746630 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402961ef-b1ce-4d0e-9169-18517d5f9285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:14.746764 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402961ef-b1ce-4d0e-9169-18517d5f9285 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:14.749347 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402961ef-b1ce-4d0e-9169-18517d5f9285 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:14.751719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402961ef-b1ce-4d0e-9169-18517d5f9285 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:14.951935 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 835/1673] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:14:14 2022] POST /identity/v3/users => generated 293 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:14.959278 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1735ca2-ed95-494e-9559-833eeb2c3833 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:14.993732 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1735ca2-ed95-494e-9559-833eeb2c3833 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:15.002613 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89d040-b060-4d06-983b-e32ef9e4fa8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:15.003646 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89d040-b060-4d06-983b-e32ef9e4fa8b admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:15.004070 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89d040-b060-4d06-983b-e32ef9e4fa8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:15.004451 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89d040-b060-4d06-983b-e32ef9e4fa8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:15.005465 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b89d040-b060-4d06-983b-e32ef9e4fa8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:15.007352 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b89d040-b060-4d06-983b-e32ef9e4fa8b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:15.047003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 839/1674] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:15.053539 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402961ef-b1ce-4d0e-9169-18517d5f9285 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:15.072051 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402961ef-b1ce-4d0e-9169-18517d5f9285 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:15.081260 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420015b7-379d-4b98-8de4-f76067df6f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:15.082470 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420015b7-379d-4b98-8de4-f76067df6f8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:15.084445 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420015b7-379d-4b98-8de4-f76067df6f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:15.085287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420015b7-379d-4b98-8de4-f76067df6f8a admin admin] PATH_INFO: `/v3/projects/fe6506d2bf8c4e508343e9e2983c5e28/users/261c6626328747a49186115df8dc8a97/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:15.114733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420015b7-379d-4b98-8de4-f76067df6f8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe6506d2bf8c4e508343e9e2983c5e28, user_id=261c6626328747a49186115df8dc8a97, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:15.124257 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420015b7-379d-4b98-8de4-f76067df6f8a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:15.242468 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 836/1675] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:14:15 2022] PUT /identity/v3/projects/fe6506d2bf8c4e508343e9e2983c5e28/users/261c6626328747a49186115df8dc8a97/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:15.250000 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89d040-b060-4d06-983b-e32ef9e4fa8b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:15.288737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89d040-b060-4d06-983b-e32ef9e4fa8b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:15.324232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420015b7-379d-4b98-8de4-f76067df6f8a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:15.470818 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70f07e-596c-4572-a649-4a44536950d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:15.471457 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70f07e-596c-4572-a649-4a44536950d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:15.471737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70f07e-596c-4572-a649-4a44536950d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:15.472010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70f07e-596c-4572-a649-4a44536950d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:15.472672 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d70f07e-596c-4572-a649-4a44536950d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:15.474315 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d70f07e-596c-4572-a649-4a44536950d7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:15.501540 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 840/1676] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:15.507785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70f07e-596c-4572-a649-4a44536950d7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:15.528178 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70f07e-596c-4572-a649-4a44536950d7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:15.545490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71983074-7af8-4092-9523-f819878baa19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:15.546246 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71983074-7af8-4092-9523-f819878baa19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:15.546562 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71983074-7af8-4092-9523-f819878baa19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:15.546841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71983074-7af8-4092-9523-f819878baa19 admin admin] PATH_INFO: `/v3/projects/fe6506d2bf8c4e508343e9e2983c5e28/users/261c6626328747a49186115df8dc8a97/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:15.552059 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71983074-7af8-4092-9523-f819878baa19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe6506d2bf8c4e508343e9e2983c5e28, user_id=261c6626328747a49186115df8dc8a97, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:15.565253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71983074-7af8-4092-9523-f819878baa19 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:15.609709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 841/1677] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:14:15 2022] PUT /identity/v3/projects/fe6506d2bf8c4e508343e9e2983c5e28/users/261c6626328747a49186115df8dc8a97/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:15.634527 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9862f3-5467-4969-a684-d63db424035f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:15.634817 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9862f3-5467-4969-a684-d63db424035f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:15.635087 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9862f3-5467-4969-a684-d63db424035f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:15.640933 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420015b7-379d-4b98-8de4-f76067df6f8a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:15.700778 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2f9862f3-5467-4969-a684-d63db424035f None None] MFA Rules not processed for user `261c6626328747a49186115df8dc8a97`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:15.702062 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2f9862f3-5467-4969-a684-d63db424035f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:15.885414 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:15.968025 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 842/1678] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:15 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:14:15.969638 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33dff60-c31a-44d5-9183-65a2ef372d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:15.971636 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33dff60-c31a-44d5-9183-65a2ef372d77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:15.972407 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33dff60-c31a-44d5-9183-65a2ef372d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:15.973115 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33dff60-c31a-44d5-9183-65a2ef372d77 admin admin] PATH_INFO: `/v3/users/ad47c1f969ae4cf4867e2e3db565e2e9` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:15.980250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9862f3-5467-4969-a684-d63db424035f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:15.983099 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33dff60-c31a-44d5-9183-65a2ef372d77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad47c1f969ae4cf4867e2e3db565e2e9)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:15.983406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2f9862f3-5467-4969-a684-d63db424035f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:15.989110 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33dff60-c31a-44d5-9183-65a2ef372d77 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:16.183078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:16.248232 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9862f3-5467-4969-a684-d63db424035f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:16.263563 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b560c884-6e24-4c53-b616-34ed679674e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:16.265115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560c884-6e24-4c53-b616-34ed679674e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:16.268130 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560c884-6e24-4c53-b616-34ed679674e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:16.268647 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560c884-6e24-4c53-b616-34ed679674e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:16.318313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b560c884-6e24-4c53-b616-34ed679674e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:16.321192 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b560c884-6e24-4c53-b616-34ed679674e6 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:16.348969 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a33dff60-c31a-44d5-9183-65a2ef372d77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad47c1f969ae4cf4867e2e3db565e2e9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:16.376040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 843/1679] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:14:16.464997 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a33dff60-c31a-44d5-9183-65a2ef372d77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad47c1f969ae4cf4867e2e3db565e2e9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:16.544986 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a33dff60-c31a-44d5-9183-65a2ef372d77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad47c1f969ae4cf4867e2e3db565e2e9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:16.546290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a33dff60-c31a-44d5-9183-65a2ef372d77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad47c1f969ae4cf4867e2e3db565e2e9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:16.547827 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a33dff60-c31a-44d5-9183-65a2ef372d77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad47c1f969ae4cf4867e2e3db565e2e9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:16.605918 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 837/1680] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:14:15 2022] DELETE /identity/v3/users/ad47c1f969ae4cf4867e2e3db565e2e9 => generated 0 bytes in 1282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:16.612555 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f058960-52b2-4f58-a8e5-a55b2fa74b9a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:16.613049 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f058960-52b2-4f58-a8e5-a55b2fa74b9a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:16.613520 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f058960-52b2-4f58-a8e5-a55b2fa74b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:16.686781 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8f058960-52b2-4f58-a8e5-a55b2fa74b9a None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:16.694564 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8f058960-52b2-4f58-a8e5-a55b2fa74b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:16.806752 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33dff60-c31a-44d5-9183-65a2ef372d77 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:16.809961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a33dff60-c31a-44d5-9183-65a2ef372d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:16.901731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 844/1681] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:16.942328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f058960-52b2-4f58-a8e5-a55b2fa74b9a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:16.944073 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8f058960-52b2-4f58-a8e5-a55b2fa74b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:16.988700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33dff60-c31a-44d5-9183-65a2ef372d77 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:16.999533 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb1a85-4374-40f7-bb14-e9414c7a61b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:17.002284 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb1a85-4374-40f7-bb14-e9414c7a61b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:17.003136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb1a85-4374-40f7-bb14-e9414c7a61b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:17.003856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb1a85-4374-40f7-bb14-e9414c7a61b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:17.010229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-67fb1a85-4374-40f7-bb14-e9414c7a61b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:17.101486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f058960-52b2-4f58-a8e5-a55b2fa74b9a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:17.121313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf335989-6cfc-460a-9f1d-a8bfa87065a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:17.122596 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf335989-6cfc-460a-9f1d-a8bfa87065a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:17.125481 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf335989-6cfc-460a-9f1d-a8bfa87065a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:17.126153 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf335989-6cfc-460a-9f1d-a8bfa87065a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:17.162054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf335989-6cfc-460a-9f1d-a8bfa87065a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:17.168426 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf335989-6cfc-460a-9f1d-a8bfa87065a5 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:17.198972 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 845/1682] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:14:17.206690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb1a85-4374-40f7-bb14-e9414c7a61b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:17.209362 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb1a85-4374-40f7-bb14-e9414c7a61b4 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:17.234442 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 838/1683] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:18.239367 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf335989-6cfc-460a-9f1d-a8bfa87065a5 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:18.246290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cf335989-6cfc-460a-9f1d-a8bfa87065a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:18.307004 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf335989-6cfc-460a-9f1d-a8bfa87065a5 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:18.343015 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd331d42-a09f-4753-b321-544e6e40dd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:18.351408 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd331d42-a09f-4753-b321-544e6e40dd29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:18.352129 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd331d42-a09f-4753-b321-544e6e40dd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:18.352752 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd331d42-a09f-4753-b321-544e6e40dd29 admin admin] PATH_INFO: `/v3/projects/c79dc0bc769c492180d10025bbea56cc` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:18.364388 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd331d42-a09f-4753-b321-544e6e40dd29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c79dc0bc769c492180d10025bbea56cc)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:18.379927 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd331d42-a09f-4753-b321-544e6e40dd29 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:18.642996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-dd331d42-a09f-4753-b321-544e6e40dd29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c79dc0bc769c492180d10025bbea56cc', 'request_id': 'req-dd331d42-a09f-4753-b321-544e6e40dd29'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:18.661938 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-dd331d42-a09f-4753-b321-544e6e40dd29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c79dc0bc769c492180d10025bbea56cc', 'request_id': 'req-dd331d42-a09f-4753-b321-544e6e40dd29'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:18.778304 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-dd331d42-a09f-4753-b321-544e6e40dd29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c79dc0bc769c492180d10025bbea56cc', 'request_id': 'req-dd331d42-a09f-4753-b321-544e6e40dd29'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:18.821540 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-dd331d42-a09f-4753-b321-544e6e40dd29 admin admin] The token cache is being invalidate because project c79dc0bc769c492180d10025bbea56cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:14:18.823261 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-dd331d42-a09f-4753-b321-544e6e40dd29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:18.824996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-dd331d42-a09f-4753-b321-544e6e40dd29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:18.835240 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 846/1684] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:14:18 2022] DELETE /identity/v3/projects/c79dc0bc769c492180d10025bbea56cc => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:18.836426 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb1a85-4374-40f7-bb14-e9414c7a61b4 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:18.868172 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb1a85-4374-40f7-bb14-e9414c7a61b4 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:18.918839 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:19.081446 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c88f39-45e5-4339-bc78-b063162268c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:19.082120 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c88f39-45e5-4339-bc78-b063162268c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:19.082444 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c88f39-45e5-4339-bc78-b063162268c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:19.082723 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c88f39-45e5-4339-bc78-b063162268c7 admin admin] PATH_INFO: `/v3/users/e09f48a88bf445e0af921b6515f8d80e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:19.085348 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c88f39-45e5-4339-bc78-b063162268c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e09f48a88bf445e0af921b6515f8d80e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:19.086137 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c88f39-45e5-4339-bc78-b063162268c7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:19.390831 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-57c88f39-45e5-4339-bc78-b063162268c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e09f48a88bf445e0af921b6515f8d80e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:19.487589 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-57c88f39-45e5-4339-bc78-b063162268c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e09f48a88bf445e0af921b6515f8d80e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:19.526437 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-57c88f39-45e5-4339-bc78-b063162268c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e09f48a88bf445e0af921b6515f8d80e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:19.529286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-57c88f39-45e5-4339-bc78-b063162268c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e09f48a88bf445e0af921b6515f8d80e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:19.530052 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-57c88f39-45e5-4339-bc78-b063162268c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e09f48a88bf445e0af921b6515f8d80e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:19.550890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 839/1685] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:14:18 2022] DELETE /identity/v3/users/e09f48a88bf445e0af921b6515f8d80e => generated 0 bytes in 715 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:20.404429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5f702-9c5e-4da7-9992-243259ccfccc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:20.405947 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5f702-9c5e-4da7-9992-243259ccfccc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:20.414656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5f702-9c5e-4da7-9992-243259ccfccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:20.505078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:20.545920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-75d5f702-9c5e-4da7-9992-243259ccfccc None None] MFA Rules not processed for user `7d2a65984c4f42fbb18977c7345c42ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:20.549417 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-75d5f702-9c5e-4da7-9992-243259ccfccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:20.902044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 847/1686] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 501 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:20.921713 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d76e8d-d52f-4411-8da5-8b481fc32a97 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:20.922965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d76e8d-d52f-4411-8da5-8b481fc32a97 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:20.924112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d76e8d-d52f-4411-8da5-8b481fc32a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:20.972610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-54d76e8d-d52f-4411-8da5-8b481fc32a97 None None] MFA Rules not processed for user `7d2a65984c4f42fbb18977c7345c42ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:20.975484 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-54d76e8d-d52f-4411-8da5-8b481fc32a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:20.989572 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 840/1687] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:21.004984 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d5f702-9c5e-4da7-9992-243259ccfccc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:21.020201 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-75d5f702-9c5e-4da7-9992-243259ccfccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:21.078507 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d76e8d-d52f-4411-8da5-8b481fc32a97 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:21.080212 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-54d76e8d-d52f-4411-8da5-8b481fc32a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:21.243204 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d5f702-9c5e-4da7-9992-243259ccfccc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:21.259905 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096c372-f48d-4bd5-b9ad-146dd8526c75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:21.262049 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096c372-f48d-4bd5-b9ad-146dd8526c75 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:21.262049 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096c372-f48d-4bd5-b9ad-146dd8526c75 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:21.262253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096c372-f48d-4bd5-b9ad-146dd8526c75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:21.283285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f096c372-f48d-4bd5-b9ad-146dd8526c75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:21.285223 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f096c372-f48d-4bd5-b9ad-146dd8526c75 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:21.312445 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d76e8d-d52f-4411-8da5-8b481fc32a97 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:21.333071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 848/1688] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:21 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:21.342261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b04c83-abd5-4656-a2d4-2b2c0978a39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:21.348302 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b04c83-abd5-4656-a2d4-2b2c0978a39d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:21.348942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b04c83-abd5-4656-a2d4-2b2c0978a39d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:21.349998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b04c83-abd5-4656-a2d4-2b2c0978a39d admin admin] PATH_INFO: `/v3/projects/14f6d7c272164667b20a3ba7727a0a13` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:21.356524 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b04c83-abd5-4656-a2d4-2b2c0978a39d admin admin] RBAC: Authorizing `identity:delete_project(project_id=14f6d7c272164667b20a3ba7727a0a13)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:21.364942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b04c83-abd5-4656-a2d4-2b2c0978a39d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:21.572902 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:21.626666 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-11b04c83-abd5-4656-a2d4-2b2c0978a39d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14f6d7c272164667b20a3ba7727a0a13', 'request_id': 'req-11b04c83-abd5-4656-a2d4-2b2c0978a39d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:21.742607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-11b04c83-abd5-4656-a2d4-2b2c0978a39d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14f6d7c272164667b20a3ba7727a0a13', 'request_id': 'req-11b04c83-abd5-4656-a2d4-2b2c0978a39d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:21.761402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-11b04c83-abd5-4656-a2d4-2b2c0978a39d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14f6d7c272164667b20a3ba7727a0a13', 'request_id': 'req-11b04c83-abd5-4656-a2d4-2b2c0978a39d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:21.794184 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-11b04c83-abd5-4656-a2d4-2b2c0978a39d admin admin] The token cache is being invalidate because project 14f6d7c272164667b20a3ba7727a0a13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:14:21.794184 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-11b04c83-abd5-4656-a2d4-2b2c0978a39d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:21.794972 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-11b04c83-abd5-4656-a2d4-2b2c0978a39d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:21.799799 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 841/1689] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:14:21 2022] DELETE /identity/v3/projects/14f6d7c272164667b20a3ba7727a0a13 => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:21.809173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc0a5f-4455-405f-b461-2265ae47ef66 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:21.810685 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc0a5f-4455-405f-b461-2265ae47ef66 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:21.811490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc0a5f-4455-405f-b461-2265ae47ef66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:21.924285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d0fc0a5f-4455-405f-b461-2265ae47ef66 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:21.934343 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d0fc0a5f-4455-405f-b461-2265ae47ef66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:22.150050 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 849/1690] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:22.159599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2aa0af-6eaa-4599-9ad0-83424b7742da None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:22.160824 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2aa0af-6eaa-4599-9ad0-83424b7742da None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:22.160824 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2aa0af-6eaa-4599-9ad0-83424b7742da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:22.195569 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ef2aa0af-6eaa-4599-9ad0-83424b7742da None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:22.197305 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ef2aa0af-6eaa-4599-9ad0-83424b7742da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:22.215765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 842/1691] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:22.221289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc0a5f-4455-405f-b461-2265ae47ef66 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:22.246828 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc0a5f-4455-405f-b461-2265ae47ef66 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:22.262028 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0eccc-2215-469a-ac34-9f76405cb5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:22.263778 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0eccc-2215-469a-ac34-9f76405cb5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:22.264172 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0eccc-2215-469a-ac34-9f76405cb5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:22.265111 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0eccc-2215-469a-ac34-9f76405cb5a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:22.265433 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e0eccc-2215-469a-ac34-9f76405cb5a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:14:22.265883 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e0eccc-2215-469a-ac34-9f76405cb5a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:22.272022 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e0eccc-2215-469a-ac34-9f76405cb5a4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:22.297040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 850/1692] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:14:22.301802 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2aa0af-6eaa-4599-9ad0-83424b7742da None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:22.325577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2aa0af-6eaa-4599-9ad0-83424b7742da None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:22.338036 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1acfbad-e7a5-4de2-bc8a-504eb46b8593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:22.339504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acfbad-e7a5-4de2-bc8a-504eb46b8593 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:22.339858 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acfbad-e7a5-4de2-bc8a-504eb46b8593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:22.345864 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acfbad-e7a5-4de2-bc8a-504eb46b8593 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:22.345864 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1acfbad-e7a5-4de2-bc8a-504eb46b8593 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:22.345864 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1acfbad-e7a5-4de2-bc8a-504eb46b8593 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:22.475082 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a6ab2-0c07-423e-ac26-6f0d9b6b637a None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:22.475733 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a6ab2-0c07-423e-ac26-6f0d9b6b637a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:22.475905 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a6ab2-0c07-423e-ac26-6f0d9b6b637a None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:22.477970 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 851/1693] 10.222.0.45 () {54 vars in 912 bytes} [Wed Aug 10 17:14:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 17:14:22.486112 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49c67c-e0c5-4c3e-a8d5-89f355e6aa84 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:22.486550 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49c67c-e0c5-4c3e-a8d5-89f355e6aa84 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:22.486933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49c67c-e0c5-4c3e-a8d5-89f355e6aa84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:22.530479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:22.662827 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 843/1694] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:14:22 2022] POST /identity/v3/projects => generated 369 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:22.670615 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1acfbad-e7a5-4de2-bc8a-504eb46b8593 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:22.711195 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0d49c67c-e0c5-4c3e-a8d5-89f355e6aa84 None None] MFA Rules not processed for user `e1534cfef9e9428bb2e010fff200bc19`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:22.732466 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0d49c67c-e0c5-4c3e-a8d5-89f355e6aa84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:22.840945 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1acfbad-e7a5-4de2-bc8a-504eb46b8593 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:22.856111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456bdac-4df1-4b82-8b51-c3ef35fa42a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:22.857573 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456bdac-4df1-4b82-8b51-c3ef35fa42a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:22.859221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456bdac-4df1-4b82-8b51-c3ef35fa42a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:22.859549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456bdac-4df1-4b82-8b51-c3ef35fa42a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:22.874618 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456bdac-4df1-4b82-8b51-c3ef35fa42a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:22.892680 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456bdac-4df1-4b82-8b51-c3ef35fa42a2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:22.989202 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 852/1695] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 10 17:14:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 504 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:23.100622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49c67c-e0c5-4c3e-a8d5-89f355e6aa84 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:23.103086 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0d49c67c-e0c5-4c3e-a8d5-89f355e6aa84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:23.124407 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49c67c-e0c5-4c3e-a8d5-89f355e6aa84 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:23.138388 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c31b73-7e6e-4910-b1c2-3ae796abcf54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:23.139130 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c31b73-7e6e-4910-b1c2-3ae796abcf54 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:23.139416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c31b73-7e6e-4910-b1c2-3ae796abcf54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:23.139785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c31b73-7e6e-4910-b1c2-3ae796abcf54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:23.159091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c31b73-7e6e-4910-b1c2-3ae796abcf54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:23.160660 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c31b73-7e6e-4910-b1c2-3ae796abcf54 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:23.180714 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 853/1696] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:14:23.347088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 844/1697] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:14:22 2022] POST /identity/v3/users => generated 285 bytes in 676 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:23.351596 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c31b73-7e6e-4910-b1c2-3ae796abcf54 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:23.401570 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c31b73-7e6e-4910-b1c2-3ae796abcf54 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:23.410404 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00697ca0-9340-408b-8d2d-110df0b2b2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:23.418460 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00697ca0-9340-408b-8d2d-110df0b2b2ba admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:23.418460 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00697ca0-9340-408b-8d2d-110df0b2b2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:23.418460 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00697ca0-9340-408b-8d2d-110df0b2b2ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:23.418460 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00697ca0-9340-408b-8d2d-110df0b2b2ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:23.422092 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00697ca0-9340-408b-8d2d-110df0b2b2ba admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:23.452567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456bdac-4df1-4b82-8b51-c3ef35fa42a2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:23.461311 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5456bdac-4df1-4b82-8b51-c3ef35fa42a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:23.480034 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 854/1698] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:14:23.485738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00697ca0-9340-408b-8d2d-110df0b2b2ba admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:23.516534 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456bdac-4df1-4b82-8b51-c3ef35fa42a2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:23.532155 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba2007-0609-4a43-ad18-f16e2016e255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:23.532928 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba2007-0609-4a43-ad18-f16e2016e255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:23.534400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba2007-0609-4a43-ad18-f16e2016e255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:23.535898 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba2007-0609-4a43-ad18-f16e2016e255 admin admin] PATH_INFO: `/v3/users/651762b4c7b24909be6fae85b2a16858` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:23.538870 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba2007-0609-4a43-ad18-f16e2016e255 admin admin] RBAC: Authorizing `identity:delete_user(user_id=651762b4c7b24909be6fae85b2a16858)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:23.546857 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba2007-0609-4a43-ad18-f16e2016e255 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:23.575871 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00697ca0-9340-408b-8d2d-110df0b2b2ba admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:23.613332 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b111575-f172-4c06-9c05-f7d1f648df51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:23.614789 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b111575-f172-4c06-9c05-f7d1f648df51 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:23.636760 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b111575-f172-4c06-9c05-f7d1f648df51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:23.639197 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b111575-f172-4c06-9c05-f7d1f648df51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:23.640770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b111575-f172-4c06-9c05-f7d1f648df51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:23.645541 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b111575-f172-4c06-9c05-f7d1f648df51 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:23.678390 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 855/1699] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:23.681282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b111575-f172-4c06-9c05-f7d1f648df51 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:23.714426 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b111575-f172-4c06-9c05-f7d1f648df51 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:23.748831 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e328855-c8fb-44ec-9624-b7f9f7bb64dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:23.754864 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e328855-c8fb-44ec-9624-b7f9f7bb64dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:23.757713 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e328855-c8fb-44ec-9624-b7f9f7bb64dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:23.758220 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e328855-c8fb-44ec-9624-b7f9f7bb64dd admin admin] PATH_INFO: `/v3/projects/0a9a1849892144db88d7a00f98d8da41/users/ebe7ec9e213d48a79fc0b543ac46eaa8/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:23.849914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-96ba2007-0609-4a43-ad18-f16e2016e255 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '651762b4c7b24909be6fae85b2a16858'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:23.872050 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e328855-c8fb-44ec-9624-b7f9f7bb64dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a9a1849892144db88d7a00f98d8da41, user_id=ebe7ec9e213d48a79fc0b543ac46eaa8, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:23.875313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e328855-c8fb-44ec-9624-b7f9f7bb64dd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:23.945486 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-96ba2007-0609-4a43-ad18-f16e2016e255 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '651762b4c7b24909be6fae85b2a16858'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:23.990999 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 856/1700] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:14:23 2022] PUT /identity/v3/projects/0a9a1849892144db88d7a00f98d8da41/users/ebe7ec9e213d48a79fc0b543ac46eaa8/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:24.014174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3184fe-ec0e-4950-8bef-1e67bf7fdd35 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:24.014490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3184fe-ec0e-4950-8bef-1e67bf7fdd35 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:24.014755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3184fe-ec0e-4950-8bef-1e67bf7fdd35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:24.023495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-96ba2007-0609-4a43-ad18-f16e2016e255 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '651762b4c7b24909be6fae85b2a16858'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:24.024529 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-96ba2007-0609-4a43-ad18-f16e2016e255 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '651762b4c7b24909be6fae85b2a16858'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:24.025527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-96ba2007-0609-4a43-ad18-f16e2016e255 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '651762b4c7b24909be6fae85b2a16858'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:24.056768 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=79,keystone:DELETE=15,keystone:INSERT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:24.095643 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-fc3184fe-ec0e-4950-8bef-1e67bf7fdd35 None None] MFA Rules not processed for user `ebe7ec9e213d48a79fc0b543ac46eaa8`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:24.100837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fc3184fe-ec0e-4950-8bef-1e67bf7fdd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:24.101491 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 845/1701] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:14:23 2022] DELETE /identity/v3/users/651762b4c7b24909be6fae85b2a16858 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:24.103056 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1cda7-8db9-4c5c-beaa-5df1dc4c4e6e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:24.115311 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1cda7-8db9-4c5c-beaa-5df1dc4c4e6e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:24.115311 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1cda7-8db9-4c5c-beaa-5df1dc4c4e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:24.188232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-38a1cda7-8db9-4c5c-beaa-5df1dc4c4e6e None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:24.195910 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-38a1cda7-8db9-4c5c-beaa-5df1dc4c4e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:24.342272 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 857/1702] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:24.344001 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3184fe-ec0e-4950-8bef-1e67bf7fdd35 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:24.345481 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fc3184fe-ec0e-4950-8bef-1e67bf7fdd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:24.477304 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 846/1703] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:24.492832 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:24.515483 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1cda7-8db9-4c5c-beaa-5df1dc4c4e6e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:24.519083 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-38a1cda7-8db9-4c5c-beaa-5df1dc4c4e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:24.568367 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3184fe-ec0e-4950-8bef-1e67bf7fdd35 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:24.585856 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdeaf92-7c5b-4c23-a278-3006f38ca8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:24.586718 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdeaf92-7c5b-4c23-a278-3006f38ca8ec admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:24.587124 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdeaf92-7c5b-4c23-a278-3006f38ca8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:24.587501 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdeaf92-7c5b-4c23-a278-3006f38ca8ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:24.588265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdeaf92-7c5b-4c23-a278-3006f38ca8ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:24.590941 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdeaf92-7c5b-4c23-a278-3006f38ca8ec admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:24.801615 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 858/1704] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:14:24 2022] POST /identity/v3/projects => generated 369 bytes in 458 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:14:24.812041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdeaf92-7c5b-4c23-a278-3006f38ca8ec admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:24.875040 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1cda7-8db9-4c5c-beaa-5df1dc4c4e6e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:25.161828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0db917-cff5-4e2f-8045-c07104cecf48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:25.162832 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0db917-cff5-4e2f-8045-c07104cecf48 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:25.170208 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0db917-cff5-4e2f-8045-c07104cecf48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:25.170208 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0db917-cff5-4e2f-8045-c07104cecf48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:25.192063 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdeaf92-7c5b-4c23-a278-3006f38ca8ec admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:25.213700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0db917-cff5-4e2f-8045-c07104cecf48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:25.213700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0db917-cff5-4e2f-8045-c07104cecf48 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:25.219650 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6261b3-4ece-48b0-b2ae-a16703e26bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:25.220720 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6261b3-4ece-48b0-b2ae-a16703e26bf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:25.221197 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6261b3-4ece-48b0-b2ae-a16703e26bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:25.221645 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6261b3-4ece-48b0-b2ae-a16703e26bf3 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:25.222716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6261b3-4ece-48b0-b2ae-a16703e26bf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:25.225408 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6261b3-4ece-48b0-b2ae-a16703e26bf3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:25.269547 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 847/1705] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 755 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:25.390997 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 859/1706] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:14:24 2022] POST /identity/v3/users => generated 291 bytes in 580 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:25.395704 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0db917-cff5-4e2f-8045-c07104cecf48 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:25.439995 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0db917-cff5-4e2f-8045-c07104cecf48 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:25.470720 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15104a4-8fb7-427a-ab68-d0d4394d5491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:25.473904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15104a4-8fb7-427a-ab68-d0d4394d5491 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:25.475202 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15104a4-8fb7-427a-ab68-d0d4394d5491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:25.476056 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15104a4-8fb7-427a-ab68-d0d4394d5491 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:25.478793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15104a4-8fb7-427a-ab68-d0d4394d5491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:25.487379 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15104a4-8fb7-427a-ab68-d0d4394d5491 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:25.535993 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6261b3-4ece-48b0-b2ae-a16703e26bf3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:25.538258 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 848/1707] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:25.576958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6261b3-4ece-48b0-b2ae-a16703e26bf3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:25.601076 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042118b-7b28-42b0-a7d2-c442aa6ce4a3 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:25.601773 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042118b-7b28-42b0-a7d2-c442aa6ce4a3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:25.602444 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042118b-7b28-42b0-a7d2-c442aa6ce4a3 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:25.607261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 849/1708] 10.222.0.45 () {54 vars in 961 bytes} [Wed Aug 10 17:14:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 17:14:25.619784 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b343ba0-3383-4151-b6f7-5fbbeb6dac85 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:25.621709 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b343ba0-3383-4151-b6f7-5fbbeb6dac85 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:25.622921 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2aaa88-667f-468b-b808-810aeb085851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:25.623599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b343ba0-3383-4151-b6f7-5fbbeb6dac85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:25.629622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2aaa88-667f-468b-b808-810aeb085851 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:25.633463 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2aaa88-667f-468b-b808-810aeb085851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:25.634104 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2aaa88-667f-468b-b808-810aeb085851 admin admin] PATH_INFO: `/v3/projects/9a03ac3a88bc437b93ce6254cab40afc/users/768c4986b1be44da8b8d852b21fd137d/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:25.716871 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2aaa88-667f-468b-b808-810aeb085851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a03ac3a88bc437b93ce6254cab40afc, user_id=768c4986b1be44da8b8d852b21fd137d, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:25.725222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6b343ba0-3383-4151-b6f7-5fbbeb6dac85 None None] MFA Rules not processed for user `8cd39d9a583a4d73a5654017be106019`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:25.733825 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2aaa88-667f-468b-b808-810aeb085851 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:25.734796 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6b343ba0-3383-4151-b6f7-5fbbeb6dac85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:25.876565 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 860/1709] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:14:25 2022] PUT /identity/v3/projects/9a03ac3a88bc437b93ce6254cab40afc/users/768c4986b1be44da8b8d852b21fd137d/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:14:25.881411 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2aaa88-667f-468b-b808-810aeb085851 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:25.939437 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:26.062802 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 850/1710] 10.222.0.45 () {58 vars in 1058 bytes} [Wed Aug 10 17:14:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 445 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:26.065273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a7439-cee2-4670-84e5-a3c5094499fc None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:26.065821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a7439-cee2-4670-84e5-a3c5094499fc None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:26.066229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a7439-cee2-4670-84e5-a3c5094499fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:26.158468 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2aaa88-667f-468b-b808-810aeb085851 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:26.173690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-225a7439-cee2-4670-84e5-a3c5094499fc None None] MFA Rules not processed for user `7d2a65984c4f42fbb18977c7345c42ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:26.175497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-225a7439-cee2-4670-84e5-a3c5094499fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:26.199932 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f649e8-7b62-4b6c-8e40-c25591457bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:26.200954 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f649e8-7b62-4b6c-8e40-c25591457bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:26.201449 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f649e8-7b62-4b6c-8e40-c25591457bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:26.201929 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f649e8-7b62-4b6c-8e40-c25591457bc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:26.203136 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f649e8-7b62-4b6c-8e40-c25591457bc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:26.219923 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f649e8-7b62-4b6c-8e40-c25591457bc4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:26.286923 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 851/1711] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:26.286923 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a7439-cee2-4670-84e5-a3c5094499fc None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:26.290894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 861/1712] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:26.290894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f649e8-7b62-4b6c-8e40-c25591457bc4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:26.327033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f649e8-7b62-4b6c-8e40-c25591457bc4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:26.329829 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe8333-b42a-4b6b-8669-2f3fb2ef5643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:26.331479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe8333-b42a-4b6b-8669-2f3fb2ef5643 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:26.331940 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe8333-b42a-4b6b-8669-2f3fb2ef5643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:26.332529 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe8333-b42a-4b6b-8669-2f3fb2ef5643 admin admin] PATH_INFO: `/v3/projects/9a03ac3a88bc437b93ce6254cab40afc/users/768c4986b1be44da8b8d852b21fd137d/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:26.336275 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe8333-b42a-4b6b-8669-2f3fb2ef5643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a03ac3a88bc437b93ce6254cab40afc, user_id=768c4986b1be44da8b8d852b21fd137d, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:26.343001 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe8333-b42a-4b6b-8669-2f3fb2ef5643 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:26.384854 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 862/1713] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:14:26 2022] PUT /identity/v3/projects/9a03ac3a88bc437b93ce6254cab40afc/users/768c4986b1be44da8b8d852b21fd137d/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:26.389733 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf283f-fc4b-471f-8384-387d17509735 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:26.389733 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf283f-fc4b-471f-8384-387d17509735 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:26.390107 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf283f-fc4b-471f-8384-387d17509735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:26.480671 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:26.508700 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e4bf283f-fc4b-471f-8384-387d17509735 None None] MFA Rules not processed for user `768c4986b1be44da8b8d852b21fd137d`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:26.514501 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e4bf283f-fc4b-471f-8384-387d17509735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:26.594223 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a7439-cee2-4670-84e5-a3c5094499fc None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:26.788475 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 863/1714] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:26 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:26.789930 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a089e1-be84-4727-90f5-220dbe0220d1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:26.790229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a089e1-be84-4727-90f5-220dbe0220d1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:26.790533 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a089e1-be84-4727-90f5-220dbe0220d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:26.834650 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc8554-29bd-437c-ac76-ca050aab3593 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cd39d9a583a4d73a5654017be106019', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:26.839720 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc8554-29bd-437c-ac76-ca050aab3593 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:26.839986 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc8554-29bd-437c-ac76-ca050aab3593 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:26.840211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc8554-29bd-437c-ac76-ca050aab3593 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:26.847834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-83bc8554-29bd-437c-ac76-ca050aab3593 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:26.872475 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-50a089e1-be84-4727-90f5-220dbe0220d1 None None] MFA Rules not processed for user `06cb7a8f70ab409b86c669620df4a651`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:26.875373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-50a089e1-be84-4727-90f5-220dbe0220d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:26.903912 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc8554-29bd-437c-ac76-ca050aab3593 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:26.906039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc8554-29bd-437c-ac76-ca050aab3593 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:26.935665 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 852/1715] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 652 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:26.936587 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc8554-29bd-437c-ac76-ca050aab3593 service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:26.940172 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-83bc8554-29bd-437c-ac76-ca050aab3593 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:26.971860 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:27.044318 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 864/1716] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:27.055880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e24620-211f-4e62-96a2-2b3248fdd28c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:27.056411 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e24620-211f-4e62-96a2-2b3248fdd28c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:27.056836 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e24620-211f-4e62-96a2-2b3248fdd28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:27.099607 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-35e24620-211f-4e62-96a2-2b3248fdd28c None None] MFA Rules not processed for user `06cb7a8f70ab409b86c669620df4a651`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:27.100879 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-35e24620-211f-4e62-96a2-2b3248fdd28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:27.122838 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 865/1717] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:14:27.134029 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e24620-211f-4e62-96a2-2b3248fdd28c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:27.135855 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-35e24620-211f-4e62-96a2-2b3248fdd28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:27.172978 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc8554-29bd-437c-ac76-ca050aab3593 service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:27.192341 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb915864-7fad-455e-8baf-28270abc03a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:27.193359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb915864-7fad-455e-8baf-28270abc03a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:27.193812 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb915864-7fad-455e-8baf-28270abc03a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:27.200786 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb915864-7fad-455e-8baf-28270abc03a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:27.256958 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb915864-7fad-455e-8baf-28270abc03a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:27.259585 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb915864-7fad-455e-8baf-28270abc03a0 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:27.366422 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 853/1718] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:26 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:27.367178 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e24620-211f-4e62-96a2-2b3248fdd28c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:27.397063 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c48e22-cf4a-41a2-bab8-ed7aa195e7dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:27.398664 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c48e22-cf4a-41a2-bab8-ed7aa195e7dc service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:27.399123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c48e22-cf4a-41a2-bab8-ed7aa195e7dc service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:27.399554 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c48e22-cf4a-41a2-bab8-ed7aa195e7dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:27.422816 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c48e22-cf4a-41a2-bab8-ed7aa195e7dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:27.424818 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c48e22-cf4a-41a2-bab8-ed7aa195e7dc service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:27.474164 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 866/1719] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:14:28.330993 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb915864-7fad-455e-8baf-28270abc03a0 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:28.333450 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-eb915864-7fad-455e-8baf-28270abc03a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:28.694860 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb915864-7fad-455e-8baf-28270abc03a0 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:28.716487 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a70134-cea8-4fd9-8df3-e1554968e1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:28.717344 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a70134-cea8-4fd9-8df3-e1554968e1ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:28.717727 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a70134-cea8-4fd9-8df3-e1554968e1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:28.718107 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a70134-cea8-4fd9-8df3-e1554968e1ef admin admin] PATH_INFO: `/v3/projects/060a1146196f42a494e830743546193e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:28.722035 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a70134-cea8-4fd9-8df3-e1554968e1ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=060a1146196f42a494e830743546193e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:28.732223 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a70134-cea8-4fd9-8df3-e1554968e1ef admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:29.020284 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f9a70134-cea8-4fd9-8df3-e1554968e1ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '060a1146196f42a494e830743546193e', 'request_id': 'req-f9a70134-cea8-4fd9-8df3-e1554968e1ef'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:29.157113 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f9a70134-cea8-4fd9-8df3-e1554968e1ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '060a1146196f42a494e830743546193e', 'request_id': 'req-f9a70134-cea8-4fd9-8df3-e1554968e1ef'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:29.165689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f9a70134-cea8-4fd9-8df3-e1554968e1ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '060a1146196f42a494e830743546193e', 'request_id': 'req-f9a70134-cea8-4fd9-8df3-e1554968e1ef'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:29.206978 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f9a70134-cea8-4fd9-8df3-e1554968e1ef admin admin] The token cache is being invalidate because project 060a1146196f42a494e830743546193e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:14:29.207276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f9a70134-cea8-4fd9-8df3-e1554968e1ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:29.210075 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f9a70134-cea8-4fd9-8df3-e1554968e1ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:29.222685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 854/1720] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:14:28 2022] DELETE /identity/v3/projects/060a1146196f42a494e830743546193e => generated 0 bytes in 889 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:29.223829 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c48e22-cf4a-41a2-bab8-ed7aa195e7dc service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:29.242048 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:29.264231 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c48e22-cf4a-41a2-bab8-ed7aa195e7dc service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:29.349938 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1630438-4a06-4e30-82e7-e72c0dbf095d None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:29.350421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1630438-4a06-4e30-82e7-e72c0dbf095d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:29.350712 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1630438-4a06-4e30-82e7-e72c0dbf095d None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:29.352259 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 855/1721] 10.222.0.45 () {54 vars in 912 bytes} [Wed Aug 10 17:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 17:14:29.367861 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6157c5e-765c-438e-bf67-e183c2fb4afc None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:29.368385 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6157c5e-765c-438e-bf67-e183c2fb4afc None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:29.368821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6157c5e-765c-438e-bf67-e183c2fb4afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:29.429434 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e6157c5e-765c-438e-bf67-e183c2fb4afc None None] MFA Rules not processed for user `e1534cfef9e9428bb2e010fff200bc19`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:29.435496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e6157c5e-765c-438e-bf67-e183c2fb4afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:29.494588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e9e74-478e-47a2-8767-15704f872dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:29.499587 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e9e74-478e-47a2-8767-15704f872dc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:29.500031 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e9e74-478e-47a2-8767-15704f872dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:29.500455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e9e74-478e-47a2-8767-15704f872dc8 admin admin] PATH_INFO: `/v3/users/15cb06cc9f8745ec8535ea7ed76a2d03` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:29.503970 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515e9e74-478e-47a2-8767-15704f872dc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15cb06cc9f8745ec8535ea7ed76a2d03)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:29.506254 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515e9e74-478e-47a2-8767-15704f872dc8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:29.635357 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:29.646843 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 856/1722] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 10 17:14:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:14:29.770704 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6157c5e-765c-438e-bf67-e183c2fb4afc None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:29.772581 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e6157c5e-765c-438e-bf67-e183c2fb4afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:29.799836 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6157c5e-765c-438e-bf67-e183c2fb4afc None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:29.808133 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2fb376-f226-4a6f-a0c7-61d625f924b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:29.831577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fb376-f226-4a6f-a0c7-61d625f924b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:29.831577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fb376-f226-4a6f-a0c7-61d625f924b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:29.831577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fb376-f226-4a6f-a0c7-61d625f924b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:29.831577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2fb376-f226-4a6f-a0c7-61d625f924b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:29.831577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2fb376-f226-4a6f-a0c7-61d625f924b9 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:29.853944 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 857/1723] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:29.861832 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e322826-1a05-4360-8a54-d991b9595842 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:29.862382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e322826-1a05-4360-8a54-d991b9595842 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:29.862659 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e322826-1a05-4360-8a54-d991b9595842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:29.911776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-515e9e74-478e-47a2-8767-15704f872dc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15cb06cc9f8745ec8535ea7ed76a2d03'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:29.939682 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5e322826-1a05-4360-8a54-d991b9595842 None None] MFA Rules not processed for user `48feb2fc02f04e739c5e1bf7d68baa16`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:29.941039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5e322826-1a05-4360-8a54-d991b9595842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:30.105326 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 858/1724] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:14:30.115658 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-515e9e74-478e-47a2-8767-15704f872dc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15cb06cc9f8745ec8535ea7ed76a2d03'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:30.170254 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-515e9e74-478e-47a2-8767-15704f872dc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15cb06cc9f8745ec8535ea7ed76a2d03'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:30.251489 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-515e9e74-478e-47a2-8767-15704f872dc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15cb06cc9f8745ec8535ea7ed76a2d03'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:30.254161 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-515e9e74-478e-47a2-8767-15704f872dc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15cb06cc9f8745ec8535ea7ed76a2d03'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:30.290306 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 867/1725] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:14:29 2022] DELETE /identity/v3/users/15cb06cc9f8745ec8535ea7ed76a2d03 => generated 0 bytes in 1067 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:14:30.555546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e322826-1a05-4360-8a54-d991b9595842 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:30.557742 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5e322826-1a05-4360-8a54-d991b9595842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:30.824146 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e322826-1a05-4360-8a54-d991b9595842 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:31.134230 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bee3a7-154b-4505-b949-f17d246369ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:31.136013 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bee3a7-154b-4505-b949-f17d246369ae service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:31.138461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bee3a7-154b-4505-b949-f17d246369ae service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:31.138916 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bee3a7-154b-4505-b949-f17d246369ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:31.149163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f9bee3a7-154b-4505-b949-f17d246369ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:31.195418 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:31.498573 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bee3a7-154b-4505-b949-f17d246369ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:31.502864 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bee3a7-154b-4505-b949-f17d246369ae service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:31.598980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 859/1726] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 1044 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:33.145019 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd5506-e775-4b83-ae1e-c70acbd98cbc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:33.150415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd5506-e775-4b83-ae1e-c70acbd98cbc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:33.152743 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd5506-e775-4b83-ae1e-c70acbd98cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:33.259179 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-92bd5506-e775-4b83-ae1e-c70acbd98cbc None None] MFA Rules not processed for user `2f74a20789dc4ef697861146252bacba`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:33.264785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-92bd5506-e775-4b83-ae1e-c70acbd98cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:33.517781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1e828-dfb9-481e-9194-28a651770700 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:33.518387 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1e828-dfb9-481e-9194-28a651770700 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:33.518712 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1e828-dfb9-481e-9194-28a651770700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:33.543829 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 868/1727] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:33.564125 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd5506-e775-4b83-ae1e-c70acbd98cbc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:33.566330 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-92bd5506-e775-4b83-ae1e-c70acbd98cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:33.617216 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd5506-e775-4b83-ae1e-c70acbd98cbc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:33.655879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-80c1e828-dfb9-481e-9194-28a651770700 None None] MFA Rules not processed for user `5bf9229cd5d6429ea14249f8976e83cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:33.658546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-80c1e828-dfb9-481e-9194-28a651770700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:33.667807 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853cb15f-4967-4586-b7a8-09dcd08fbc25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:33.668523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cb15f-4967-4586-b7a8-09dcd08fbc25 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:33.668785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cb15f-4967-4586-b7a8-09dcd08fbc25 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:33.669045 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cb15f-4967-4586-b7a8-09dcd08fbc25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:33.701989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853cb15f-4967-4586-b7a8-09dcd08fbc25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:33.704906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853cb15f-4967-4586-b7a8-09dcd08fbc25 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:33.739496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:33.760420 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 869/1728] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:14:33.847090 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853cb15f-4967-4586-b7a8-09dcd08fbc25 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:33.849141 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-853cb15f-4967-4586-b7a8-09dcd08fbc25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:34.056502 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 860/1729] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:33 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 541 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:34.072467 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853cb15f-4967-4586-b7a8-09dcd08fbc25 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:34.072815 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1e828-dfb9-481e-9194-28a651770700 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:34.075429 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-80c1e828-dfb9-481e-9194-28a651770700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:34.098977 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7839df-ac3f-4673-829e-ab18cf4efd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:34.098977 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7839df-ac3f-4673-829e-ab18cf4efd16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:34.098977 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7839df-ac3f-4673-829e-ab18cf4efd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:34.098977 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7839df-ac3f-4673-829e-ab18cf4efd16 admin admin] PATH_INFO: `/v3/projects/d578d2c4c6da4e00a386db0eb012b071` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:34.099375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7839df-ac3f-4673-829e-ab18cf4efd16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d578d2c4c6da4e00a386db0eb012b071)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:34.112402 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7839df-ac3f-4673-829e-ab18cf4efd16 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:34.201726 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1e828-dfb9-481e-9194-28a651770700 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:34.244993 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32a569-2468-42da-8838-f124b5cbf508 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:34.247573 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32a569-2468-42da-8838-f124b5cbf508 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:34.252399 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32a569-2468-42da-8838-f124b5cbf508 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:34.254117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32a569-2468-42da-8838-f124b5cbf508 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:34.299610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a32a569-2468-42da-8838-f124b5cbf508 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:34.301794 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a32a569-2468-42da-8838-f124b5cbf508 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:34.334446 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 861/1730] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:34 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:14:34.498271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7e7839df-ac3f-4673-829e-ab18cf4efd16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd578d2c4c6da4e00a386db0eb012b071', 'request_id': 'req-7e7839df-ac3f-4673-829e-ab18cf4efd16'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:34.542834 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7e7839df-ac3f-4673-829e-ab18cf4efd16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd578d2c4c6da4e00a386db0eb012b071', 'request_id': 'req-7e7839df-ac3f-4673-829e-ab18cf4efd16'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:34.700420 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7e7839df-ac3f-4673-829e-ab18cf4efd16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd578d2c4c6da4e00a386db0eb012b071', 'request_id': 'req-7e7839df-ac3f-4673-829e-ab18cf4efd16'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:34.744473 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7e7839df-ac3f-4673-829e-ab18cf4efd16 admin admin] The token cache is being invalidate because project d578d2c4c6da4e00a386db0eb012b071 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:14:34.745301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7e7839df-ac3f-4673-829e-ab18cf4efd16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:34.750742 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7e7839df-ac3f-4673-829e-ab18cf4efd16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:34.757180 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 870/1731] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:14:33 2022] DELETE /identity/v3/projects/d578d2c4c6da4e00a386db0eb012b071 => generated 0 bytes in 910 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:34.765360 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e620e41-9b4c-4bff-99f1-b21e8e658f17 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:34.767082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e620e41-9b4c-4bff-99f1-b21e8e658f17 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:34.767556 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e620e41-9b4c-4bff-99f1-b21e8e658f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:34.832053 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7e620e41-9b4c-4bff-99f1-b21e8e658f17 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:34.838062 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7e620e41-9b4c-4bff-99f1-b21e8e658f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:34.934075 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7839df-ac3f-4673-829e-ab18cf4efd16 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:34.935962 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7e7839df-ac3f-4673-829e-ab18cf4efd16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:35.191363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 862/1732] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:35.199568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28690631-f498-47d9-8e30-67c4ff326e16 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:35.199568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28690631-f498-47d9-8e30-67c4ff326e16 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:35.199568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28690631-f498-47d9-8e30-67c4ff326e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:35.236814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7839df-ac3f-4673-829e-ab18cf4efd16 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:35.261785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166317d0-0942-4074-8bb7-b42e48bc84d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:35.262506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166317d0-0942-4074-8bb7-b42e48bc84d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:35.263056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166317d0-0942-4074-8bb7-b42e48bc84d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:35.263335 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166317d0-0942-4074-8bb7-b42e48bc84d6 admin admin] PATH_INFO: `/v3/users/ebe7ec9e213d48a79fc0b543ac46eaa8` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:35.268468 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166317d0-0942-4074-8bb7-b42e48bc84d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebe7ec9e213d48a79fc0b543ac46eaa8)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:35.272876 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166317d0-0942-4074-8bb7-b42e48bc84d6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:35.388872 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:35.413798 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-28690631-f498-47d9-8e30-67c4ff326e16 None None] MFA Rules not processed for user `4f9e1767ec90435f80248a2930081265`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:35.421050 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-28690631-f498-47d9-8e30-67c4ff326e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:35.625949 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:35.650733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 863/1733] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 10 17:14:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:35.656863 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341dc180-d4b9-4149-9e53-5bb9b802d969 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:35.657259 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341dc180-d4b9-4149-9e53-5bb9b802d969 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:35.657414 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341dc180-d4b9-4149-9e53-5bb9b802d969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:35.666983 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-166317d0-0942-4074-8bb7-b42e48bc84d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebe7ec9e213d48a79fc0b543ac46eaa8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:35.693593 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-341dc180-d4b9-4149-9e53-5bb9b802d969 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:35.695089 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-341dc180-d4b9-4149-9e53-5bb9b802d969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:35.710915 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 864/1734] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:35.716613 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341dc180-d4b9-4149-9e53-5bb9b802d969 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:35.729185 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341dc180-d4b9-4149-9e53-5bb9b802d969 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:35.741108 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12705f7-3d65-4890-83d2-9fd1866405b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:35.741903 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12705f7-3d65-4890-83d2-9fd1866405b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:35.742297 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12705f7-3d65-4890-83d2-9fd1866405b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:35.743791 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12705f7-3d65-4890-83d2-9fd1866405b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:35.745128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12705f7-3d65-4890-83d2-9fd1866405b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:14:35.745608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12705f7-3d65-4890-83d2-9fd1866405b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:35.749167 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12705f7-3d65-4890-83d2-9fd1866405b1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:35.777349 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 865/1735] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 17:14:35.786662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12705f7-3d65-4890-83d2-9fd1866405b1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:35.790542 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-166317d0-0942-4074-8bb7-b42e48bc84d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebe7ec9e213d48a79fc0b543ac46eaa8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:35.806503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-166317d0-0942-4074-8bb7-b42e48bc84d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebe7ec9e213d48a79fc0b543ac46eaa8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:35.830057 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12705f7-3d65-4890-83d2-9fd1866405b1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:35.846942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814757e-4a24-4414-89d4-3a2e2732fc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:35.847575 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814757e-4a24-4414-89d4-3a2e2732fc17 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:35.847854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814757e-4a24-4414-89d4-3a2e2732fc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:35.848124 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814757e-4a24-4414-89d4-3a2e2732fc17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:35.848766 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814757e-4a24-4414-89d4-3a2e2732fc17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:35.850768 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814757e-4a24-4414-89d4-3a2e2732fc17 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:35.872776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-166317d0-0942-4074-8bb7-b42e48bc84d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebe7ec9e213d48a79fc0b543ac46eaa8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:35.875017 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-166317d0-0942-4074-8bb7-b42e48bc84d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebe7ec9e213d48a79fc0b543ac46eaa8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:35.894064 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 871/1736] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:14:34 2022] DELETE /identity/v3/users/ebe7ec9e213d48a79fc0b543ac46eaa8 => generated 0 bytes in 963 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:14:35.904162 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e442007-9094-45d1-ab42-0e5f046ab9fb None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:35.904162 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e442007-9094-45d1-ab42-0e5f046ab9fb None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:35.904162 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e442007-9094-45d1-ab42-0e5f046ab9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:35.967551 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3e442007-9094-45d1-ab42-0e5f046ab9fb None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:35.970965 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3e442007-9094-45d1-ab42-0e5f046ab9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:36.011495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 866/1737] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:14:35 2022] POST /identity/v3/projects => generated 409 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:36.020111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814757e-4a24-4414-89d4-3a2e2732fc17 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:36.023942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9814757e-4a24-4414-89d4-3a2e2732fc17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:36.067599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 872/1738] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:36.109646 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e442007-9094-45d1-ab42-0e5f046ab9fb None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:36.117300 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3e442007-9094-45d1-ab42-0e5f046ab9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:36.201305 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814757e-4a24-4414-89d4-3a2e2732fc17 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:36.214594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aedf31-3197-4293-b418-e0ce17854717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:36.215243 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aedf31-3197-4293-b418-e0ce17854717 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:36.215513 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aedf31-3197-4293-b418-e0ce17854717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:36.215785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aedf31-3197-4293-b418-e0ce17854717 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:36.216640 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aedf31-3197-4293-b418-e0ce17854717 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:36.221408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aedf31-3197-4293-b418-e0ce17854717 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:36.340526 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e442007-9094-45d1-ab42-0e5f046ab9fb None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:36.343943 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 867/1739] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:14:36 2022] POST /identity/v3/users => generated 305 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:36.344959 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aedf31-3197-4293-b418-e0ce17854717 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:36.356062 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-97aedf31-3197-4293-b418-e0ce17854717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:36.560461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:36.567313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb7f525-66f5-49c6-aef4-e000368aaa0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:36.567876 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7f525-66f5-49c6-aef4-e000368aaa0d service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:36.568156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7f525-66f5-49c6-aef4-e000368aaa0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:36.568428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7f525-66f5-49c6-aef4-e000368aaa0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:36.601345 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb7f525-66f5-49c6-aef4-e000368aaa0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:36.602953 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb7f525-66f5-49c6-aef4-e000368aaa0d service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:36.669223 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 873/1740] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:36.675907 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb7f525-66f5-49c6-aef4-e000368aaa0d service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:36.708268 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb7f525-66f5-49c6-aef4-e000368aaa0d service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:36.714390 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aedf31-3197-4293-b418-e0ce17854717 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:36.725438 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37518f52-7757-4807-ab9d-0e154ec1db2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:36.726531 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37518f52-7757-4807-ab9d-0e154ec1db2e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:36.726941 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37518f52-7757-4807-ab9d-0e154ec1db2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:36.727363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37518f52-7757-4807-ab9d-0e154ec1db2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:36.728798 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37518f52-7757-4807-ab9d-0e154ec1db2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:36.731779 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37518f52-7757-4807-ab9d-0e154ec1db2e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:36.736479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:36.743080 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3b302-4522-4cd7-9edb-4eeadae7974b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4986821ef9f84b99a835e387635aa9a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:36.750650 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3b302-4522-4cd7-9edb-4eeadae7974b service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:36.750650 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3b302-4522-4cd7-9edb-4eeadae7974b service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:36.750650 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3b302-4522-4cd7-9edb-4eeadae7974b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:36.754141 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-14e3b302-4522-4cd7-9edb-4eeadae7974b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:36.759485 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 874/1741] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:36.764206 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37518f52-7757-4807-ab9d-0e154ec1db2e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:36.786312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37518f52-7757-4807-ab9d-0e154ec1db2e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:36.807756 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb37bc1-b416-41ec-a31f-936a929c3374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:36.808290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb37bc1-b416-41ec-a31f-936a929c3374 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:36.808571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb37bc1-b416-41ec-a31f-936a929c3374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:36.808839 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb37bc1-b416-41ec-a31f-936a929c3374 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:36.809498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb37bc1-b416-41ec-a31f-936a929c3374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:36.811328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb37bc1-b416-41ec-a31f-936a929c3374 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:36.843784 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 875/1742] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:36.848171 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb37bc1-b416-41ec-a31f-936a929c3374 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:36.878429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb37bc1-b416-41ec-a31f-936a929c3374 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:36.891490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29979218-e0dd-4258-b562-250666c3f6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:36.893692 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29979218-e0dd-4258-b562-250666c3f6fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:36.893996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29979218-e0dd-4258-b562-250666c3f6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:36.894273 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29979218-e0dd-4258-b562-250666c3f6fa admin admin] PATH_INFO: `/v3/projects/49e6f60c3a614ccbafef669c22950d0c/users/c3da39491e914ece85307ce0cf70c02e/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:36.937505 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3b302-4522-4cd7-9edb-4eeadae7974b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:36.950742 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3b302-4522-4cd7-9edb-4eeadae7974b service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:36.962304 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29979218-e0dd-4258-b562-250666c3f6fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=49e6f60c3a614ccbafef669c22950d0c, user_id=c3da39491e914ece85307ce0cf70c02e, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:36.969814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29979218-e0dd-4258-b562-250666c3f6fa admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:36.981661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 868/1743] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 10 17:14:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 17:14:37.148298 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 876/1744] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:14:36 2022] PUT /identity/v3/projects/49e6f60c3a614ccbafef669c22950d0c/users/c3da39491e914ece85307ce0cf70c02e/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:37.159217 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28888156-0da7-4f33-be6c-8719778cc84a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:37.160188 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28888156-0da7-4f33-be6c-8719778cc84a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:37.160473 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28888156-0da7-4f33-be6c-8719778cc84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:37.237981 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-28888156-0da7-4f33-be6c-8719778cc84a None None] MFA Rules not processed for user `c3da39491e914ece85307ce0cf70c02e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:37.245770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-28888156-0da7-4f33-be6c-8719778cc84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:37.484740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 869/1745] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:37 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:14:37.488366 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29979218-e0dd-4258-b562-250666c3f6fa admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:37.693334 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29979218-e0dd-4258-b562-250666c3f6fa admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:37.705688 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc33e82-42ac-440b-896c-3a4fe6dcbd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:37.706799 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc33e82-42ac-440b-896c-3a4fe6dcbd8f admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:37.707273 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc33e82-42ac-440b-896c-3a4fe6dcbd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:37.709756 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc33e82-42ac-440b-896c-3a4fe6dcbd8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:37.713208 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc33e82-42ac-440b-896c-3a4fe6dcbd8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:37.720534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc33e82-42ac-440b-896c-3a4fe6dcbd8f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:37.906487 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 877/1746] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:14:37 2022] POST /identity/v3/projects => generated 409 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:37.907432 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28888156-0da7-4f33-be6c-8719778cc84a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:37.926009 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28888156-0da7-4f33-be6c-8719778cc84a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:38.102675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:38.156588 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69eeec-0a9e-4b18-a94d-ad396a7ecd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:38.159811 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69eeec-0a9e-4b18-a94d-ad396a7ecd10 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:38.167150 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69eeec-0a9e-4b18-a94d-ad396a7ecd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:38.173841 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69eeec-0a9e-4b18-a94d-ad396a7ecd10 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:38.176353 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69eeec-0a9e-4b18-a94d-ad396a7ecd10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:38.195993 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69eeec-0a9e-4b18-a94d-ad396a7ecd10 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:38.602467 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 870/1747] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:14:37 2022] POST /identity/v3/users => generated 311 bytes in 695 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:38.608121 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc33e82-42ac-440b-896c-3a4fe6dcbd8f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:38.656868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc33e82-42ac-440b-896c-3a4fe6dcbd8f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:38.683046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2b92f-0d9f-40bc-b91d-b1f6eacccefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:38.693555 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2b92f-0d9f-40bc-b91d-b1f6eacccefb admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:38.700079 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2b92f-0d9f-40bc-b91d-b1f6eacccefb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:38.700636 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2b92f-0d9f-40bc-b91d-b1f6eacccefb admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:38.708442 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc2b92f-0d9f-40bc-b91d-b1f6eacccefb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:38.716186 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc2b92f-0d9f-40bc-b91d-b1f6eacccefb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:38.814923 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69eeec-0a9e-4b18-a94d-ad396a7ecd10 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:38.824903 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 878/1748] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:38.863077 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69eeec-0a9e-4b18-a94d-ad396a7ecd10 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:38.883394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcbe1c3-da56-4827-a651-223f47bee938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:38.886899 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbe1c3-da56-4827-a651-223f47bee938 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:38.886899 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbe1c3-da56-4827-a651-223f47bee938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:38.886899 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbe1c3-da56-4827-a651-223f47bee938 admin admin] PATH_INFO: `/v3/projects/028feb93133d467d9f6fd1ea7b53bcc3/users/df461c8ab2ac483fad0d71d72e65b34a/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:38.940896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcbe1c3-da56-4827-a651-223f47bee938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=028feb93133d467d9f6fd1ea7b53bcc3, user_id=df461c8ab2ac483fad0d71d72e65b34a, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:38.947397 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcbe1c3-da56-4827-a651-223f47bee938 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:39.090495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 871/1749] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:14:38 2022] PUT /identity/v3/projects/028feb93133d467d9f6fd1ea7b53bcc3/users/df461c8ab2ac483fad0d71d72e65b34a/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:39.090809 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2b92f-0d9f-40bc-b91d-b1f6eacccefb admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:39.118110 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2b92f-0d9f-40bc-b91d-b1f6eacccefb admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:39.164959 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392fbc1-0a33-4767-96c0-872af9ee434a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:39.165526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392fbc1-0a33-4767-96c0-872af9ee434a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:39.165526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392fbc1-0a33-4767-96c0-872af9ee434a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:39.215079 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0392fbc1-0a33-4767-96c0-872af9ee434a None None] MFA Rules not processed for user `e1534cfef9e9428bb2e010fff200bc19`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:39.216335 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0392fbc1-0a33-4767-96c0-872af9ee434a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:39.308669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cfe7f-8d44-4dc5-87d5-44bf25b2d6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:39.309276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cfe7f-8d44-4dc5-87d5-44bf25b2d6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:39.310485 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cfe7f-8d44-4dc5-87d5-44bf25b2d6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:39.310485 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cfe7f-8d44-4dc5-87d5-44bf25b2d6d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:39.310843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3cfe7f-8d44-4dc5-87d5-44bf25b2d6d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:39.312174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3cfe7f-8d44-4dc5-87d5-44bf25b2d6d5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:39.360925 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 879/1750] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:39.370332 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cfe7f-8d44-4dc5-87d5-44bf25b2d6d5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:39.421768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cfe7f-8d44-4dc5-87d5-44bf25b2d6d5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:39.441661 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9684580b-4758-4b32-81ac-a204b283635c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:39.441661 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684580b-4758-4b32-81ac-a204b283635c admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:39.441661 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684580b-4758-4b32-81ac-a204b283635c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:39.442234 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684580b-4758-4b32-81ac-a204b283635c admin admin] PATH_INFO: `/v3/projects/028feb93133d467d9f6fd1ea7b53bcc3/users/df461c8ab2ac483fad0d71d72e65b34a/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:39.447111 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9684580b-4758-4b32-81ac-a204b283635c admin admin] RBAC: Authorizing `identity:create_grant(project_id=028feb93133d467d9f6fd1ea7b53bcc3, user_id=df461c8ab2ac483fad0d71d72e65b34a, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:39.450112 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9684580b-4758-4b32-81ac-a204b283635c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:39.451686 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 872/1751] 10.222.0.49 () {58 vars in 1015 bytes} [Wed Aug 10 17:14:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:39.492647 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 880/1752] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:14:39 2022] PUT /identity/v3/projects/028feb93133d467d9f6fd1ea7b53bcc3/users/df461c8ab2ac483fad0d71d72e65b34a/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:39.496178 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427ebba-a34d-4f53-b5f6-4aaa81cfa849 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:39.498808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427ebba-a34d-4f53-b5f6-4aaa81cfa849 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:39.499336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427ebba-a34d-4f53-b5f6-4aaa81cfa849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:39.501955 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9684580b-4758-4b32-81ac-a204b283635c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:39.506517 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9684580b-4758-4b32-81ac-a204b283635c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:39.596190 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:39.611740 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3427ebba-a34d-4f53-b5f6-4aaa81cfa849 None None] MFA Rules not processed for user `df461c8ab2ac483fad0d71d72e65b34a`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:39.613507 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3427ebba-a34d-4f53-b5f6-4aaa81cfa849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:39.793738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9684580b-4758-4b32-81ac-a204b283635c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:39.815889 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db894245-496d-4476-8a1d-5730b2cbc480 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:39.816503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db894245-496d-4476-8a1d-5730b2cbc480 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:39.816787 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db894245-496d-4476-8a1d-5730b2cbc480 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:39.817079 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db894245-496d-4476-8a1d-5730b2cbc480 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:39.854869 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db894245-496d-4476-8a1d-5730b2cbc480 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:39.856405 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db894245-496d-4476-8a1d-5730b2cbc480 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:39.897209 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 881/1753] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:40.121264 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 873/1754] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:39 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:40.122276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db894245-496d-4476-8a1d-5730b2cbc480 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:40.122276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-db894245-496d-4476-8a1d-5730b2cbc480 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:40.231482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db894245-496d-4476-8a1d-5730b2cbc480 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:40.258881 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9cfcc9-146c-4cc8-a54f-9ca7958e4298 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:40.260115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cfcc9-146c-4cc8-a54f-9ca7958e4298 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:40.263635 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cfcc9-146c-4cc8-a54f-9ca7958e4298 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:40.263897 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cfcc9-146c-4cc8-a54f-9ca7958e4298 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:40.333423 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9cfcc9-146c-4cc8-a54f-9ca7958e4298 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:40.344003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9cfcc9-146c-4cc8-a54f-9ca7958e4298 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:40.394372 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 882/1755] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:39 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:40.998707 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427ebba-a34d-4f53-b5f6-4aaa81cfa849 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:41.001308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3427ebba-a34d-4f53-b5f6-4aaa81cfa849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:41.191105 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:41.208175 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427ebba-a34d-4f53-b5f6-4aaa81cfa849 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:41.223451 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f584e-1230-4753-9e2b-3100f5fa14ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:41.224228 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f584e-1230-4753-9e2b-3100f5fa14ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:41.224513 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f584e-1230-4753-9e2b-3100f5fa14ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:41.224788 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f584e-1230-4753-9e2b-3100f5fa14ac admin admin] PATH_INFO: `/v3/projects/0a9a1849892144db88d7a00f98d8da41` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:41.231636 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f584e-1230-4753-9e2b-3100f5fa14ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a9a1849892144db88d7a00f98d8da41)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:41.235260 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f584e-1230-4753-9e2b-3100f5fa14ac admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:41.401980 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9cfcc9-146c-4cc8-a54f-9ca7958e4298 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:41.405669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ff9cfcc9-146c-4cc8-a54f-9ca7958e4298 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:41.483329 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9e7f584e-1230-4753-9e2b-3100f5fa14ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a9a1849892144db88d7a00f98d8da41', 'request_id': 'req-9e7f584e-1230-4753-9e2b-3100f5fa14ac'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:41.484093 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9cfcc9-146c-4cc8-a54f-9ca7958e4298 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:41.687380 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03327d7-0bd2-4c13-abd1-dba5b67fa3c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:41.688130 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03327d7-0bd2-4c13-abd1-dba5b67fa3c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:41.688528 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03327d7-0bd2-4c13-abd1-dba5b67fa3c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:41.688906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03327d7-0bd2-4c13-abd1-dba5b67fa3c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:41.755053 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9e7f584e-1230-4753-9e2b-3100f5fa14ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a9a1849892144db88d7a00f98d8da41', 'request_id': 'req-9e7f584e-1230-4753-9e2b-3100f5fa14ac'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:41.768933 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9e7f584e-1230-4753-9e2b-3100f5fa14ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a9a1849892144db88d7a00f98d8da41', 'request_id': 'req-9e7f584e-1230-4753-9e2b-3100f5fa14ac'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:41.820415 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9e7f584e-1230-4753-9e2b-3100f5fa14ac admin admin] The token cache is being invalidate because project 0a9a1849892144db88d7a00f98d8da41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:14:41.820746 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9e7f584e-1230-4753-9e2b-3100f5fa14ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:41.822614 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9e7f584e-1230-4753-9e2b-3100f5fa14ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:41.830631 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 874/1756] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:14:40 2022] DELETE /identity/v3/projects/0a9a1849892144db88d7a00f98d8da41 => generated 0 bytes in 832 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:41.831609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03327d7-0bd2-4c13-abd1-dba5b67fa3c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:41.833168 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03327d7-0bd2-4c13-abd1-dba5b67fa3c9 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:41.837850 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f584e-1230-4753-9e2b-3100f5fa14ac admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:41.859218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9e7f584e-1230-4753-9e2b-3100f5fa14ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:41.859400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 883/1757] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:41 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:42.059357 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f584e-1230-4753-9e2b-3100f5fa14ac admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:42.071696 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402da8d5-89ee-4096-837e-237950c41402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:42.073649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402da8d5-89ee-4096-837e-237950c41402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:42.074336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402da8d5-89ee-4096-837e-237950c41402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:42.075751 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402da8d5-89ee-4096-837e-237950c41402 admin admin] PATH_INFO: `/v3/users/768c4986b1be44da8b8d852b21fd137d` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:42.078327 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402da8d5-89ee-4096-837e-237950c41402 admin admin] RBAC: Authorizing `identity:delete_user(user_id=768c4986b1be44da8b8d852b21fd137d)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:42.099418 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402da8d5-89ee-4096-837e-237950c41402 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:42.563913 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-402da8d5-89ee-4096-837e-237950c41402 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '768c4986b1be44da8b8d852b21fd137d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:42.737364 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-402da8d5-89ee-4096-837e-237950c41402 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '768c4986b1be44da8b8d852b21fd137d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:42.819466 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-402da8d5-89ee-4096-837e-237950c41402 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '768c4986b1be44da8b8d852b21fd137d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:42.823205 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-402da8d5-89ee-4096-837e-237950c41402 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '768c4986b1be44da8b8d852b21fd137d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:42.826743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-402da8d5-89ee-4096-837e-237950c41402 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '768c4986b1be44da8b8d852b21fd137d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:42.897961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 875/1758] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:14:41 2022] DELETE /identity/v3/users/768c4986b1be44da8b8d852b21fd137d => generated 0 bytes in 1062 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:46.911791 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdf0db-c5a1-423a-a5c0-60a89892fc05 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:46.914514 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdf0db-c5a1-423a-a5c0-60a89892fc05 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:46.915507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdf0db-c5a1-423a-a5c0-60a89892fc05 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:46.919981 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 884/1759] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 10 17:14:46 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:14:46.954663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53493c6f-ffc7-443d-8498-b9220a7a7aa3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:46.960955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53493c6f-ffc7-443d-8498-b9220a7a7aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:46.960955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53493c6f-ffc7-443d-8498-b9220a7a7aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:47.029026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:47.033835 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdf0db-c5a1-423a-a5c0-60a89892fc05 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:47.038122 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7dfdf0db-c5a1-423a-a5c0-60a89892fc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:47.107078 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-53493c6f-ffc7-443d-8498-b9220a7a7aa3 None None] MFA Rules not processed for user `615cc36e21cc493b823e6cee695ac308`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:47.109415 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-53493c6f-ffc7-443d-8498-b9220a7a7aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:47.259710 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:47.389369 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 876/1760] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 10 17:14:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 437 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:47.459185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdf0db-c5a1-423a-a5c0-60a89892fc05 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:47.477908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab45ce7-2874-4dcd-a3be-fad3e8743b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:47.479879 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab45ce7-2874-4dcd-a3be-fad3e8743b30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:47.480352 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab45ce7-2874-4dcd-a3be-fad3e8743b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:47.480782 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab45ce7-2874-4dcd-a3be-fad3e8743b30 admin admin] PATH_INFO: `/v3/projects/9a03ac3a88bc437b93ce6254cab40afc` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:47.487974 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab45ce7-2874-4dcd-a3be-fad3e8743b30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a03ac3a88bc437b93ce6254cab40afc)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:47.520810 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab45ce7-2874-4dcd-a3be-fad3e8743b30 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:47.780179 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5ab45ce7-2874-4dcd-a3be-fad3e8743b30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a03ac3a88bc437b93ce6254cab40afc', 'request_id': 'req-5ab45ce7-2874-4dcd-a3be-fad3e8743b30'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:47.798962 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5ab45ce7-2874-4dcd-a3be-fad3e8743b30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a03ac3a88bc437b93ce6254cab40afc', 'request_id': 'req-5ab45ce7-2874-4dcd-a3be-fad3e8743b30'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:47.950580 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5ab45ce7-2874-4dcd-a3be-fad3e8743b30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a03ac3a88bc437b93ce6254cab40afc', 'request_id': 'req-5ab45ce7-2874-4dcd-a3be-fad3e8743b30'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:47.983240 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5ab45ce7-2874-4dcd-a3be-fad3e8743b30 admin admin] The token cache is being invalidate because project 9a03ac3a88bc437b93ce6254cab40afc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:14:47.987013 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5ab45ce7-2874-4dcd-a3be-fad3e8743b30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:47.988146 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5ab45ce7-2874-4dcd-a3be-fad3e8743b30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:47.991239 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 885/1761] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:14:47 2022] DELETE /identity/v3/projects/9a03ac3a88bc437b93ce6254cab40afc => generated 0 bytes in 958 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:48.016247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90aa43-38e7-4fef-9f91-c402a2096f7d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:48.028427 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90aa43-38e7-4fef-9f91-c402a2096f7d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:48.028427 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90aa43-38e7-4fef-9f91-c402a2096f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:48.028792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33427402-158c-435c-91f0-fae7fd2dc75d None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:48.028792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33427402-158c-435c-91f0-fae7fd2dc75d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:48.028792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33427402-158c-435c-91f0-fae7fd2dc75d None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:48.028792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 877/1762] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 10 17:14:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:14:48.107564 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3d90aa43-38e7-4fef-9f91-c402a2096f7d None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:48.110544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3d90aa43-38e7-4fef-9f91-c402a2096f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:48.325317 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 886/1763] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:48.334992 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff22ae-7a31-4934-8028-8d3c5bce560d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:48.334992 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff22ae-7a31-4934-8028-8d3c5bce560d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:48.335315 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff22ae-7a31-4934-8028-8d3c5bce560d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:48.382597 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6fff22ae-7a31-4934-8028-8d3c5bce560d None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:48.384225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6fff22ae-7a31-4934-8028-8d3c5bce560d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:48.409376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 878/1764] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:48.417314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90aa43-38e7-4fef-9f91-c402a2096f7d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:48.427798 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd942fa-e7ce-4cf5-867e-57e1ae38bf50 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:48.428070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd942fa-e7ce-4cf5-867e-57e1ae38bf50 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:48.428528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd942fa-e7ce-4cf5-867e-57e1ae38bf50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:48.481779 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3cd942fa-e7ce-4cf5-867e-57e1ae38bf50 None None] MFA Rules not processed for user `615cc36e21cc493b823e6cee695ac308`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:48.486271 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3cd942fa-e7ce-4cf5-867e-57e1ae38bf50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:48.499839 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90aa43-38e7-4fef-9f91-c402a2096f7d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:48.528588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323507b-3f14-4773-8e26-bf2b24c36e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:48.531182 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323507b-3f14-4773-8e26-bf2b24c36e7c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:48.531182 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323507b-3f14-4773-8e26-bf2b24c36e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:48.531765 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323507b-3f14-4773-8e26-bf2b24c36e7c admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:48.544387 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323507b-3f14-4773-8e26-bf2b24c36e7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:14:48.544387 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323507b-3f14-4773-8e26-bf2b24c36e7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:48.544387 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323507b-3f14-4773-8e26-bf2b24c36e7c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:48.668503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 887/1765] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:14:48.674497 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323507b-3f14-4773-8e26-bf2b24c36e7c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:48.695511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323507b-3f14-4773-8e26-bf2b24c36e7c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:48.708789 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24117fc-0526-47d1-bbdf-fb4d4fc6eff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:48.709421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24117fc-0526-47d1-bbdf-fb4d4fc6eff5 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:48.709718 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24117fc-0526-47d1-bbdf-fb4d4fc6eff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:48.709955 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24117fc-0526-47d1-bbdf-fb4d4fc6eff5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:48.710627 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24117fc-0526-47d1-bbdf-fb4d4fc6eff5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:48.712589 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24117fc-0526-47d1-bbdf-fb4d4fc6eff5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:48.724928 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 879/1766] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 10 17:14:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:48.966888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 888/1767] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:14:48 2022] POST /identity/v3/projects => generated 393 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:48.989741 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd942fa-e7ce-4cf5-867e-57e1ae38bf50 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:49.014007 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd942fa-e7ce-4cf5-867e-57e1ae38bf50 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:49.130987 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837367b7-0051-4acb-bce7-65c5d0eef37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:49.131931 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837367b7-0051-4acb-bce7-65c5d0eef37c admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:49.131931 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837367b7-0051-4acb-bce7-65c5d0eef37c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:49.132196 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837367b7-0051-4acb-bce7-65c5d0eef37c admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:49.132676 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837367b7-0051-4acb-bce7-65c5d0eef37c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:49.136130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837367b7-0051-4acb-bce7-65c5d0eef37c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:49.200560 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24117fc-0526-47d1-bbdf-fb4d4fc6eff5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:49.204754 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e24117fc-0526-47d1-bbdf-fb4d4fc6eff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:49.273304 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 880/1768] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:14:48 2022] POST /identity/v3/users => generated 297 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:49.276016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837367b7-0051-4acb-bce7-65c5d0eef37c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:49.279123 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-837367b7-0051-4acb-bce7-65c5d0eef37c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:49.429302 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24117fc-0526-47d1-bbdf-fb4d4fc6eff5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:49.441325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:49.453021 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e57a29-40a4-470d-8544-a053db6cce89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:49.455748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e57a29-40a4-470d-8544-a053db6cce89 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:49.456569 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e57a29-40a4-470d-8544-a053db6cce89 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:49.457284 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e57a29-40a4-470d-8544-a053db6cce89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:49.462334 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c5e57a29-40a4-470d-8544-a053db6cce89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:49.471237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837367b7-0051-4acb-bce7-65c5d0eef37c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:49.485182 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497073a-507e-4e1b-8e99-03c4783fe0db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:49.486815 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497073a-507e-4e1b-8e99-03c4783fe0db service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:49.487793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497073a-507e-4e1b-8e99-03c4783fe0db service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:49.488264 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497073a-507e-4e1b-8e99-03c4783fe0db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:49.499807 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e57a29-40a4-470d-8544-a053db6cce89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:49.505497 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e57a29-40a4-470d-8544-a053db6cce89 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:49.512373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497073a-507e-4e1b-8e99-03c4783fe0db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:49.516234 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497073a-507e-4e1b-8e99-03c4783fe0db service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:49.535503 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 881/1769] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:49.539294 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497073a-507e-4e1b-8e99-03c4783fe0db service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:49.550826 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 889/1770] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:49.552908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988076f8-2981-4071-8308-f389aea5e6db None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:49.553343 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988076f8-2981-4071-8308-f389aea5e6db None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:49.553835 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988076f8-2981-4071-8308-f389aea5e6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:49.562632 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497073a-507e-4e1b-8e99-03c4783fe0db service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:49.580964 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb62c3-3c34-4e71-9f5e-39cf9d1dd353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:49.582971 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb62c3-3c34-4e71-9f5e-39cf9d1dd353 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:49.583454 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb62c3-3c34-4e71-9f5e-39cf9d1dd353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:49.583927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb62c3-3c34-4e71-9f5e-39cf9d1dd353 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:49.585374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fb62c3-3c34-4e71-9f5e-39cf9d1dd353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:49.589224 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fb62c3-3c34-4e71-9f5e-39cf9d1dd353 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:49.601163 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-988076f8-2981-4071-8308-f389aea5e6db None None] MFA Rules not processed for user `48feb2fc02f04e739c5e1bf7d68baa16`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:49.602696 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-988076f8-2981-4071-8308-f389aea5e6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:49.610748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 882/1771] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:49.616701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb62c3-3c34-4e71-9f5e-39cf9d1dd353 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:49.667869 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb62c3-3c34-4e71-9f5e-39cf9d1dd353 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:49.688478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aede892-5ba2-45ae-9a95-f91a3e90b90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:49.689578 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aede892-5ba2-45ae-9a95-f91a3e90b90f admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:49.690155 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aede892-5ba2-45ae-9a95-f91a3e90b90f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:49.694629 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aede892-5ba2-45ae-9a95-f91a3e90b90f admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:49.694629 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aede892-5ba2-45ae-9a95-f91a3e90b90f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:49.698613 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aede892-5ba2-45ae-9a95-f91a3e90b90f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:49.752309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 883/1772] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:49.754658 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aede892-5ba2-45ae-9a95-f91a3e90b90f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:49.771272 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aede892-5ba2-45ae-9a95-f91a3e90b90f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:49.781672 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 890/1773] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:49.792790 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265fade-15a5-4a5b-9831-d3f55cb138d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:49.792790 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265fade-15a5-4a5b-9831-d3f55cb138d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:49.792790 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265fade-15a5-4a5b-9831-d3f55cb138d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:49.792790 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265fade-15a5-4a5b-9831-d3f55cb138d2 admin admin] PATH_INFO: `/v3/projects/66236e5ec6ea4d4592b0af56771c5d0d/users/cd6a46ec814c42cd904fc373039c2627/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:49.817670 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265fade-15a5-4a5b-9831-d3f55cb138d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66236e5ec6ea4d4592b0af56771c5d0d, user_id=cd6a46ec814c42cd904fc373039c2627, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:49.826037 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265fade-15a5-4a5b-9831-d3f55cb138d2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:49.884023 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f3d8f-8273-4a51-9e42-214b53fc1a22 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:49.884023 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f3d8f-8273-4a51-9e42-214b53fc1a22 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:49.884023 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f3d8f-8273-4a51-9e42-214b53fc1a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:49.946726 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 884/1774] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:14:49 2022] PUT /identity/v3/projects/66236e5ec6ea4d4592b0af56771c5d0d/users/cd6a46ec814c42cd904fc373039c2627/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:14:49.961934 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd43d8-a9d1-4536-87e0-0d8091606907 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:49.962475 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd43d8-a9d1-4536-87e0-0d8091606907 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:49.963433 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f52f3d8f-8273-4a51-9e42-214b53fc1a22 None None] MFA Rules not processed for user `2f74a20789dc4ef697861146252bacba`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:49.963987 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd43d8-a9d1-4536-87e0-0d8091606907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:49.966188 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f52f3d8f-8273-4a51-9e42-214b53fc1a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:50.029488 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-00fd43d8-a9d1-4536-87e0-0d8091606907 None None] MFA Rules not processed for user `cd6a46ec814c42cd904fc373039c2627`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:50.033546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-00fd43d8-a9d1-4536-87e0-0d8091606907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:50.085753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 891/1775] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:50.215163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 885/1776] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:49 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:14:50.222500 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f3d8f-8273-4a51-9e42-214b53fc1a22 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:50.387368 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f3d8f-8273-4a51-9e42-214b53fc1a22 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:50.397965 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50806fbd-c363-4039-bc13-e6c502063afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:50.402030 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50806fbd-c363-4039-bc13-e6c502063afc admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:50.404061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50806fbd-c363-4039-bc13-e6c502063afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:50.404781 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50806fbd-c363-4039-bc13-e6c502063afc admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:50.411506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50806fbd-c363-4039-bc13-e6c502063afc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:50.411506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50806fbd-c363-4039-bc13-e6c502063afc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:50.550524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 892/1777] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:14:50 2022] POST /identity/v3/projects => generated 393 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:50.554742 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd43d8-a9d1-4536-87e0-0d8091606907 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:50.566675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd43d8-a9d1-4536-87e0-0d8091606907 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:50.695531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314c128-6c9f-4b1c-97c2-b74aecdaf106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:50.696723 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314c128-6c9f-4b1c-97c2-b74aecdaf106 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:50.697142 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314c128-6c9f-4b1c-97c2-b74aecdaf106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:50.698139 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314c128-6c9f-4b1c-97c2-b74aecdaf106 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:50.699328 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3314c128-6c9f-4b1c-97c2-b74aecdaf106 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:50.703305 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3314c128-6c9f-4b1c-97c2-b74aecdaf106 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:50.815680 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 886/1778] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:14:50 2022] POST /identity/v3/users => generated 303 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:14:50.819616 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50806fbd-c363-4039-bc13-e6c502063afc admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:50.838320 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50806fbd-c363-4039-bc13-e6c502063afc admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:50.846798 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f536dbe-0413-4ce1-b7fb-66176b71d28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:50.847830 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f536dbe-0413-4ce1-b7fb-66176b71d28c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:50.848441 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f536dbe-0413-4ce1-b7fb-66176b71d28c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:50.849064 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f536dbe-0413-4ce1-b7fb-66176b71d28c admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:50.850494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f536dbe-0413-4ce1-b7fb-66176b71d28c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:50.853430 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f536dbe-0413-4ce1-b7fb-66176b71d28c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:50.872709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 893/1779] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:14:50.878493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314c128-6c9f-4b1c-97c2-b74aecdaf106 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:50.902923 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314c128-6c9f-4b1c-97c2-b74aecdaf106 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:50.911686 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e49d6-4d55-48cb-b775-125ca6de7cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:50.913080 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e49d6-4d55-48cb-b775-125ca6de7cda admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:50.913579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e49d6-4d55-48cb-b775-125ca6de7cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:50.914087 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e49d6-4d55-48cb-b775-125ca6de7cda admin admin] PATH_INFO: `/v3/projects/f9bbf53915f840049c3e306423921d63/users/83ea6f5e9ec348268cee23d79345e65e/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:50.969731 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e49d6-4d55-48cb-b775-125ca6de7cda admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9bbf53915f840049c3e306423921d63, user_id=83ea6f5e9ec348268cee23d79345e65e, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:50.977663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e49d6-4d55-48cb-b775-125ca6de7cda admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:51.101601 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 887/1780] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:14:50 2022] PUT /identity/v3/projects/f9bbf53915f840049c3e306423921d63/users/83ea6f5e9ec348268cee23d79345e65e/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:14:51.106799 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4dfc6-5e42-48bd-91c7-f35ce47f4148 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:51.107536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4dfc6-5e42-48bd-91c7-f35ce47f4148 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:51.107883 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e49d6-4d55-48cb-b775-125ca6de7cda admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:51.108005 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4dfc6-5e42-48bd-91c7-f35ce47f4148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:51.175500 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d5f4dfc6-5e42-48bd-91c7-f35ce47f4148 None None] MFA Rules not processed for user `5bf9229cd5d6429ea14249f8976e83cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:51.180353 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d5f4dfc6-5e42-48bd-91c7-f35ce47f4148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:51.266607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e49d6-4d55-48cb-b775-125ca6de7cda admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:51.345187 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981d66f-6be8-4cac-922e-1638b6994bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:51.347712 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 894/1781] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:51.349481 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981d66f-6be8-4cac-922e-1638b6994bea admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:51.352653 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981d66f-6be8-4cac-922e-1638b6994bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:51.353047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981d66f-6be8-4cac-922e-1638b6994bea admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:51.356584 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981d66f-6be8-4cac-922e-1638b6994bea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:51.367145 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981d66f-6be8-4cac-922e-1638b6994bea admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:51.410493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 888/1782] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:14:51.415655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4dfc6-5e42-48bd-91c7-f35ce47f4148 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:51.458999 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4dfc6-5e42-48bd-91c7-f35ce47f4148 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:51.465825 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:51.481312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b4da9-f81c-4fee-9303-28247253e916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:51.481312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b4da9-f81c-4fee-9303-28247253e916 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:51.482718 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b4da9-f81c-4fee-9303-28247253e916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:51.483528 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b4da9-f81c-4fee-9303-28247253e916 admin admin] PATH_INFO: `/v3/projects/f9bbf53915f840049c3e306423921d63/users/83ea6f5e9ec348268cee23d79345e65e/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:51.518427 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b4da9-f81c-4fee-9303-28247253e916 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9bbf53915f840049c3e306423921d63, user_id=83ea6f5e9ec348268cee23d79345e65e, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:51.529767 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b4da9-f81c-4fee-9303-28247253e916 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:51.594009 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 895/1783] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:14:51 2022] PUT /identity/v3/projects/f9bbf53915f840049c3e306423921d63/users/83ea6f5e9ec348268cee23d79345e65e/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:14:51.616691 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d57e9-5ddc-41c6-a90d-71b0c307e3b5 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:51.617167 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d57e9-5ddc-41c6-a90d-71b0c307e3b5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:51.617599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d57e9-5ddc-41c6-a90d-71b0c307e3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:51.761996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ee2d57e9-5ddc-41c6-a90d-71b0c307e3b5 None None] MFA Rules not processed for user `83ea6f5e9ec348268cee23d79345e65e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:51.770307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ee2d57e9-5ddc-41c6-a90d-71b0c307e3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:52.201354 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 889/1784] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:51 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 586 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:14:52.225719 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b4da9-f81c-4fee-9303-28247253e916 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:52.229788 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0f9b4da9-f81c-4fee-9303-28247253e916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:52.502521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b4da9-f81c-4fee-9303-28247253e916 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:52.511340 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1305479-a1f0-458c-af54-b0037fb7b84a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:52.514312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1305479-a1f0-458c-af54-b0037fb7b84a service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:52.515532 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1305479-a1f0-458c-af54-b0037fb7b84a service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:52.516486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1305479-a1f0-458c-af54-b0037fb7b84a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:52.580174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1305479-a1f0-458c-af54-b0037fb7b84a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:52.581720 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1305479-a1f0-458c-af54-b0037fb7b84a service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:52.613738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 896/1785] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:52 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:14:52.679925 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:52.884442 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d57e9-5ddc-41c6-a90d-71b0c307e3b5 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:52.918902 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d57e9-5ddc-41c6-a90d-71b0c307e3b5 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:52.949186 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8746d2d-3513-430a-be9d-39cd2505baa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:52.950608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8746d2d-3513-430a-be9d-39cd2505baa8 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:52.952584 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8746d2d-3513-430a-be9d-39cd2505baa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:52.953099 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8746d2d-3513-430a-be9d-39cd2505baa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:53.231571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8746d2d-3513-430a-be9d-39cd2505baa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:53.233200 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8746d2d-3513-430a-be9d-39cd2505baa8 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:53.307726 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 890/1786] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:52 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:14:54.253824 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:57.380571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:58.929765 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1305479-a1f0-458c-af54-b0037fb7b84a service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:58.932331 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b1305479-a1f0-458c-af54-b0037fb7b84a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:59.018700 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1305479-a1f0-458c-af54-b0037fb7b84a service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:59.024757 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8af590-60c2-4a98-bd60-b547e40edcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:59.026244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8af590-60c2-4a98-bd60-b547e40edcc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:59.026569 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8af590-60c2-4a98-bd60-b547e40edcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:59.027030 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8af590-60c2-4a98-bd60-b547e40edcc2 admin admin] PATH_INFO: `/v3/users/c3da39491e914ece85307ce0cf70c02e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:59.028373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8af590-60c2-4a98-bd60-b547e40edcc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3da39491e914ece85307ce0cf70c02e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:59.030430 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8af590-60c2-4a98-bd60-b547e40edcc2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:59.033193 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:14:59.318128 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3d8af590-60c2-4a98-bd60-b547e40edcc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3da39491e914ece85307ce0cf70c02e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:59.428640 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3d8af590-60c2-4a98-bd60-b547e40edcc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3da39491e914ece85307ce0cf70c02e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:59.441310 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3d8af590-60c2-4a98-bd60-b547e40edcc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3da39491e914ece85307ce0cf70c02e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:59.460108 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3d8af590-60c2-4a98-bd60-b547e40edcc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3da39491e914ece85307ce0cf70c02e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:59.461201 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3d8af590-60c2-4a98-bd60-b547e40edcc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3da39491e914ece85307ce0cf70c02e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:14:59.477741 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 897/1787] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:14:58 2022] DELETE /identity/v3/users/c3da39491e914ece85307ce0cf70c02e => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:14:59.480833 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0e049-28a3-4ea1-a174-1ea2c81a4c17 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:59.481977 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0e049-28a3-4ea1-a174-1ea2c81a4c17 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:59.482470 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0e049-28a3-4ea1-a174-1ea2c81a4c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:59.521287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-77a0e049-28a3-4ea1-a174-1ea2c81a4c17 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:14:59.523551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-77a0e049-28a3-4ea1-a174-1ea2c81a4c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:59.633558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 891/1788] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:14:59.648887 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8af590-60c2-4a98-bd60-b547e40edcc2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:14:59.651206 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3d8af590-60c2-4a98-bd60-b547e40edcc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:14:59.727017 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8af590-60c2-4a98-bd60-b547e40edcc2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:14:59.730892 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5310022-7616-4b0d-a8e8-a96ccdfe8915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:14:59.731512 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5310022-7616-4b0d-a8e8-a96ccdfe8915 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:14:59.731790 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5310022-7616-4b0d-a8e8-a96ccdfe8915 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:14:59.732061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5310022-7616-4b0d-a8e8-a96ccdfe8915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:14:59.742215 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5310022-7616-4b0d-a8e8-a96ccdfe8915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:14:59.745256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5310022-7616-4b0d-a8e8-a96ccdfe8915 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:14:59.756348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 898/1789] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:14:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:15:00.510465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0e049-28a3-4ea1-a174-1ea2c81a4c17 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:00.515067 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-77a0e049-28a3-4ea1-a174-1ea2c81a4c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:00.542472 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0e049-28a3-4ea1-a174-1ea2c81a4c17 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:00.557133 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc67a9-27f9-4367-a828-9a51dffbb65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:00.558192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc67a9-27f9-4367-a828-9a51dffbb65a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:00.559175 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc67a9-27f9-4367-a828-9a51dffbb65a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:00.559662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc67a9-27f9-4367-a828-9a51dffbb65a admin admin] PATH_INFO: `/v3/projects/49e6f60c3a614ccbafef669c22950d0c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:00.562236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc67a9-27f9-4367-a828-9a51dffbb65a admin admin] RBAC: Authorizing `identity:delete_project(project_id=49e6f60c3a614ccbafef669c22950d0c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:00.568403 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc67a9-27f9-4367-a828-9a51dffbb65a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:00.743352 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b0dc67a9-27f9-4367-a828-9a51dffbb65a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49e6f60c3a614ccbafef669c22950d0c', 'request_id': 'req-b0dc67a9-27f9-4367-a828-9a51dffbb65a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:00.921787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b0dc67a9-27f9-4367-a828-9a51dffbb65a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49e6f60c3a614ccbafef669c22950d0c', 'request_id': 'req-b0dc67a9-27f9-4367-a828-9a51dffbb65a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:00.929069 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b0dc67a9-27f9-4367-a828-9a51dffbb65a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49e6f60c3a614ccbafef669c22950d0c', 'request_id': 'req-b0dc67a9-27f9-4367-a828-9a51dffbb65a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:00.941273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b0dc67a9-27f9-4367-a828-9a51dffbb65a admin admin] The token cache is being invalidate because project 49e6f60c3a614ccbafef669c22950d0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:15:00.941541 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b0dc67a9-27f9-4367-a828-9a51dffbb65a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:00.942147 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b0dc67a9-27f9-4367-a828-9a51dffbb65a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:00.944130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 892/1790] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:15:00 2022] DELETE /identity/v3/projects/49e6f60c3a614ccbafef669c22950d0c => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:00.948677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5310022-7616-4b0d-a8e8-a96ccdfe8915 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:00.959021 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5310022-7616-4b0d-a8e8-a96ccdfe8915 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:01.021520 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf140100-b6ea-449d-bf01-1ce2db7d3008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:01.022145 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf140100-b6ea-449d-bf01-1ce2db7d3008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:01.022460 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf140100-b6ea-449d-bf01-1ce2db7d3008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:01.022776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf140100-b6ea-449d-bf01-1ce2db7d3008 admin admin] PATH_INFO: `/v3/users/df461c8ab2ac483fad0d71d72e65b34a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:01.023912 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf140100-b6ea-449d-bf01-1ce2db7d3008 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df461c8ab2ac483fad0d71d72e65b34a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:01.026460 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf140100-b6ea-449d-bf01-1ce2db7d3008 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:01.050732 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:01.237027 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf140100-b6ea-449d-bf01-1ce2db7d3008 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df461c8ab2ac483fad0d71d72e65b34a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:01.372755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf140100-b6ea-449d-bf01-1ce2db7d3008 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df461c8ab2ac483fad0d71d72e65b34a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:01.384604 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf140100-b6ea-449d-bf01-1ce2db7d3008 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df461c8ab2ac483fad0d71d72e65b34a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:01.405384 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf140100-b6ea-449d-bf01-1ce2db7d3008 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df461c8ab2ac483fad0d71d72e65b34a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:01.406280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf140100-b6ea-449d-bf01-1ce2db7d3008 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df461c8ab2ac483fad0d71d72e65b34a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:01.415807 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 899/1791] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:15:00 2022] DELETE /identity/v3/users/df461c8ab2ac483fad0d71d72e65b34a => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:15:02.025910 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc67a9-27f9-4367-a828-9a51dffbb65a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:02.029308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b0dc67a9-27f9-4367-a828-9a51dffbb65a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:02.062203 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc67a9-27f9-4367-a828-9a51dffbb65a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:02.142197 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:02.153053 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce0cba-f87f-43f7-b455-dbceb93e2387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:02.153701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce0cba-f87f-43f7-b455-dbceb93e2387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:02.153973 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce0cba-f87f-43f7-b455-dbceb93e2387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:02.154265 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce0cba-f87f-43f7-b455-dbceb93e2387 admin admin] PATH_INFO: `/v3/projects/028feb93133d467d9f6fd1ea7b53bcc3` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:02.156032 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce0cba-f87f-43f7-b455-dbceb93e2387 admin admin] RBAC: Authorizing `identity:delete_project(project_id=028feb93133d467d9f6fd1ea7b53bcc3)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:02.157963 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce0cba-f87f-43f7-b455-dbceb93e2387 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:02.267951 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-32ce0cba-f87f-43f7-b455-dbceb93e2387 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '028feb93133d467d9f6fd1ea7b53bcc3', 'request_id': 'req-32ce0cba-f87f-43f7-b455-dbceb93e2387'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:02.333115 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-32ce0cba-f87f-43f7-b455-dbceb93e2387 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '028feb93133d467d9f6fd1ea7b53bcc3', 'request_id': 'req-32ce0cba-f87f-43f7-b455-dbceb93e2387'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:02.342556 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-32ce0cba-f87f-43f7-b455-dbceb93e2387 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '028feb93133d467d9f6fd1ea7b53bcc3', 'request_id': 'req-32ce0cba-f87f-43f7-b455-dbceb93e2387'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:02.356473 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-32ce0cba-f87f-43f7-b455-dbceb93e2387 admin admin] The token cache is being invalidate because project 028feb93133d467d9f6fd1ea7b53bcc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:15:02.356942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-32ce0cba-f87f-43f7-b455-dbceb93e2387 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:02.357639 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-32ce0cba-f87f-43f7-b455-dbceb93e2387 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:02.359409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 893/1792] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:15:02 2022] DELETE /identity/v3/projects/028feb93133d467d9f6fd1ea7b53bcc3 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:02.365005 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39229fe8-bfc8-4671-96ca-d66969bd4cbd None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:02.365597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39229fe8-bfc8-4671-96ca-d66969bd4cbd None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:02.365597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39229fe8-bfc8-4671-96ca-d66969bd4cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:02.389137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-39229fe8-bfc8-4671-96ca-d66969bd4cbd None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:02.390479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-39229fe8-bfc8-4671-96ca-d66969bd4cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:02.459972 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 900/1793] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:15:02.469028 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156044b-a10c-4270-a362-b01f2fe1b3dc None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:02.469356 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156044b-a10c-4270-a362-b01f2fe1b3dc None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:02.469673 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156044b-a10c-4270-a362-b01f2fe1b3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:02.492773 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-9156044b-a10c-4270-a362-b01f2fe1b3dc None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:02.494731 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9156044b-a10c-4270-a362-b01f2fe1b3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:02.503256 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 894/1794] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:02.507876 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39229fe8-bfc8-4671-96ca-d66969bd4cbd None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:02.518928 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39229fe8-bfc8-4671-96ca-d66969bd4cbd None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:02.523904 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642ca76-88aa-4b55-aa87-84ba96da0905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:02.524531 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642ca76-88aa-4b55-aa87-84ba96da0905 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:02.524810 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642ca76-88aa-4b55-aa87-84ba96da0905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:02.525081 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642ca76-88aa-4b55-aa87-84ba96da0905 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:02.525747 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642ca76-88aa-4b55-aa87-84ba96da0905 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:15:02.526044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642ca76-88aa-4b55-aa87-84ba96da0905 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:02.527731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642ca76-88aa-4b55-aa87-84ba96da0905 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:02.540680 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 901/1795] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 17:15:02.550868 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156044b-a10c-4270-a362-b01f2fe1b3dc None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:02.561646 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156044b-a10c-4270-a362-b01f2fe1b3dc None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:02.566890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5aa0c5-0b13-44b8-ba9a-6cefbc110481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:02.567635 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5aa0c5-0b13-44b8-ba9a-6cefbc110481 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:02.567972 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5aa0c5-0b13-44b8-ba9a-6cefbc110481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:02.568354 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5aa0c5-0b13-44b8-ba9a-6cefbc110481 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:02.569075 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5aa0c5-0b13-44b8-ba9a-6cefbc110481 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:02.571707 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5aa0c5-0b13-44b8-ba9a-6cefbc110481 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:02.673730 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 895/1796] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:15:02 2022] POST /identity/v3/projects => generated 371 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:15:02.678276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642ca76-88aa-4b55-aa87-84ba96da0905 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:02.689625 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642ca76-88aa-4b55-aa87-84ba96da0905 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:02.766221 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c7401-459b-4a0b-bd26-09e0e4cd4358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:02.767103 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c7401-459b-4a0b-bd26-09e0e4cd4358 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:02.767471 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c7401-459b-4a0b-bd26-09e0e4cd4358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:02.767844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c7401-459b-4a0b-bd26-09e0e4cd4358 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:02.768632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111c7401-459b-4a0b-bd26-09e0e4cd4358 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:02.770771 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111c7401-459b-4a0b-bd26-09e0e4cd4358 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:02.889451 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 902/1797] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:15:02 2022] POST /identity/v3/users => generated 286 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:15:02.894960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5aa0c5-0b13-44b8-ba9a-6cefbc110481 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:02.907776 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5aa0c5-0b13-44b8-ba9a-6cefbc110481 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:02.915641 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e8c6c-15d4-48a0-b0cf-92f6c61e7b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:02.917555 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e8c6c-15d4-48a0-b0cf-92f6c61e7b64 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:02.917824 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e8c6c-15d4-48a0-b0cf-92f6c61e7b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:02.918417 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e8c6c-15d4-48a0-b0cf-92f6c61e7b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:02.920062 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e8c6c-15d4-48a0-b0cf-92f6c61e7b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:02.922972 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e8c6c-15d4-48a0-b0cf-92f6c61e7b64 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:02.944083 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 896/1798] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:15:02.950970 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c7401-459b-4a0b-bd26-09e0e4cd4358 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:02.967975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c7401-459b-4a0b-bd26-09e0e4cd4358 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:02.977364 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db845761-7f77-4249-a87a-49a5e1fab0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:02.979169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db845761-7f77-4249-a87a-49a5e1fab0c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:02.979512 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db845761-7f77-4249-a87a-49a5e1fab0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:02.979909 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db845761-7f77-4249-a87a-49a5e1fab0c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:02.980836 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db845761-7f77-4249-a87a-49a5e1fab0c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:02.982981 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db845761-7f77-4249-a87a-49a5e1fab0c1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:03.002617 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 903/1799] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:15:03.005133 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e8c6c-15d4-48a0-b0cf-92f6c61e7b64 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:03.016781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e8c6c-15d4-48a0-b0cf-92f6c61e7b64 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:03.021994 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c72af-4ecb-4b23-ac97-e36edff75c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:03.022698 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c72af-4ecb-4b23-ac97-e36edff75c1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:03.022970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c72af-4ecb-4b23-ac97-e36edff75c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:03.023252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c72af-4ecb-4b23-ac97-e36edff75c1f admin admin] PATH_INFO: `/v3/projects/3bd86e43a6d341ff8c699033bf09beb2/users/3c1391d78825449d9f4e8a729e4b89b4/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:03.047806 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c72af-4ecb-4b23-ac97-e36edff75c1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bd86e43a6d341ff8c699033bf09beb2, user_id=3c1391d78825449d9f4e8a729e4b89b4, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:03.050593 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c72af-4ecb-4b23-ac97-e36edff75c1f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:03.140251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 897/1800] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:15:03 2022] PUT /identity/v3/projects/3bd86e43a6d341ff8c699033bf09beb2/users/3c1391d78825449d9f4e8a729e4b89b4/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:15:03.152098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f7195-8225-49a7-a603-c9c74f0abe6a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:03.152624 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f7195-8225-49a7-a603-c9c74f0abe6a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:03.152923 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f7195-8225-49a7-a603-c9c74f0abe6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:03.170110 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:03.194785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-238f7195-8225-49a7-a603-c9c74f0abe6a None None] MFA Rules not processed for user `3c1391d78825449d9f4e8a729e4b89b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:03.196919 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-238f7195-8225-49a7-a603-c9c74f0abe6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:03.317544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 904/1801] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:03.331320 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c72af-4ecb-4b23-ac97-e36edff75c1f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:03.333428 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6f9c72af-4ecb-4b23-ac97-e36edff75c1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:03.428297 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c72af-4ecb-4b23-ac97-e36edff75c1f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:03.432411 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e79ed-9f24-4aad-99b9-e2f95136f955 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:03.433034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e79ed-9f24-4aad-99b9-e2f95136f955 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:03.433313 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e79ed-9f24-4aad-99b9-e2f95136f955 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:03.433580 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e79ed-9f24-4aad-99b9-e2f95136f955 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:03.443409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e79ed-9f24-4aad-99b9-e2f95136f955 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:03.447793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e79ed-9f24-4aad-99b9-e2f95136f955 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:03.458529 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 898/1802] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:15:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:15:03.489817 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f7195-8225-49a7-a603-c9c74f0abe6a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:03.601569 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f7195-8225-49a7-a603-c9c74f0abe6a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:03.610083 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42315dca-09cc-491a-84f6-e44841913088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:03.619436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42315dca-09cc-491a-84f6-e44841913088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:03.619436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42315dca-09cc-491a-84f6-e44841913088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:03.619436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42315dca-09cc-491a-84f6-e44841913088 admin admin] PATH_INFO: `/v3/users/3c1391d78825449d9f4e8a729e4b89b4` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:03.619436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42315dca-09cc-491a-84f6-e44841913088 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c1391d78825449d9f4e8a729e4b89b4)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:03.619436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42315dca-09cc-491a-84f6-e44841913088 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:03.881523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-42315dca-09cc-491a-84f6-e44841913088 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c1391d78825449d9f4e8a729e4b89b4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:03.955404 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-42315dca-09cc-491a-84f6-e44841913088 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c1391d78825449d9f4e8a729e4b89b4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:03.967359 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-42315dca-09cc-491a-84f6-e44841913088 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c1391d78825449d9f4e8a729e4b89b4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:03.991580 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-42315dca-09cc-491a-84f6-e44841913088 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c1391d78825449d9f4e8a729e4b89b4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:03.991580 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-42315dca-09cc-491a-84f6-e44841913088 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c1391d78825449d9f4e8a729e4b89b4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:03.995544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 905/1803] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:15:03 2022] DELETE /identity/v3/users/3c1391d78825449d9f4e8a729e4b89b4 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:04.000565 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20e070-80f4-4629-9e5b-6214b5be4172 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:04.000906 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20e070-80f4-4629-9e5b-6214b5be4172 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:04.001314 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20e070-80f4-4629-9e5b-6214b5be4172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:04.023018 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-bc20e070-80f4-4629-9e5b-6214b5be4172 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:04.024262 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bc20e070-80f4-4629-9e5b-6214b5be4172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:04.039775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:04.095498 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 899/1804] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:04.106753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42315dca-09cc-491a-84f6-e44841913088 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:04.108491 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-42315dca-09cc-491a-84f6-e44841913088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:04.133672 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:04.201057 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42315dca-09cc-491a-84f6-e44841913088 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:04.205933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a580dc-0435-430e-a368-d23eef9d4fea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:04.209935 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a580dc-0435-430e-a368-d23eef9d4fea service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:04.210412 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a580dc-0435-430e-a368-d23eef9d4fea service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:04.210802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a580dc-0435-430e-a368-d23eef9d4fea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:04.232935 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a580dc-0435-430e-a368-d23eef9d4fea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:04.235088 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a580dc-0435-430e-a368-d23eef9d4fea service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:04.249653 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 906/1805] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:15:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:15:05.034150 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20e070-80f4-4629-9e5b-6214b5be4172 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:05.038885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bc20e070-80f4-4629-9e5b-6214b5be4172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:05.063729 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20e070-80f4-4629-9e5b-6214b5be4172 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:05.072599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5860df-b581-4cf0-848a-056dfd2894a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:05.073891 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5860df-b581-4cf0-848a-056dfd2894a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:05.074377 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5860df-b581-4cf0-848a-056dfd2894a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:05.074882 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5860df-b581-4cf0-848a-056dfd2894a0 admin admin] PATH_INFO: `/v3/projects/3bd86e43a6d341ff8c699033bf09beb2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:05.077263 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5860df-b581-4cf0-848a-056dfd2894a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bd86e43a6d341ff8c699033bf09beb2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:05.080796 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5860df-b581-4cf0-848a-056dfd2894a0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:05.240274 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2d5860df-b581-4cf0-848a-056dfd2894a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bd86e43a6d341ff8c699033bf09beb2', 'request_id': 'req-2d5860df-b581-4cf0-848a-056dfd2894a0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:05.367617 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2d5860df-b581-4cf0-848a-056dfd2894a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bd86e43a6d341ff8c699033bf09beb2', 'request_id': 'req-2d5860df-b581-4cf0-848a-056dfd2894a0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:05.377334 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2d5860df-b581-4cf0-848a-056dfd2894a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bd86e43a6d341ff8c699033bf09beb2', 'request_id': 'req-2d5860df-b581-4cf0-848a-056dfd2894a0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:05.393437 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2d5860df-b581-4cf0-848a-056dfd2894a0 admin admin] The token cache is being invalidate because project 3bd86e43a6d341ff8c699033bf09beb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:15:05.393924 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2d5860df-b581-4cf0-848a-056dfd2894a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:05.397128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2d5860df-b581-4cf0-848a-056dfd2894a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:05.399893 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 900/1806] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:15:05 2022] DELETE /identity/v3/projects/3bd86e43a6d341ff8c699033bf09beb2 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:05.407492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176ac28-49d2-4cd1-9139-35b2d49eecd2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:05.408436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176ac28-49d2-4cd1-9139-35b2d49eecd2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:05.408436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176ac28-49d2-4cd1-9139-35b2d49eecd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:05.434466 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3176ac28-49d2-4cd1-9139-35b2d49eecd2 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:05.435819 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3176ac28-49d2-4cd1-9139-35b2d49eecd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:05.521300 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 907/1807] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:05.529898 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71ffec-4231-4ea5-94d1-98792ba55b36 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:05.530307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71ffec-4231-4ea5-94d1-98792ba55b36 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:05.530751 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71ffec-4231-4ea5-94d1-98792ba55b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:05.560619 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ca71ffec-4231-4ea5-94d1-98792ba55b36 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:05.566897 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ca71ffec-4231-4ea5-94d1-98792ba55b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:05.574892 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 901/1808] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:05.580716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176ac28-49d2-4cd1-9139-35b2d49eecd2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:05.599360 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176ac28-49d2-4cd1-9139-35b2d49eecd2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:05.611753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259cb6fc-1844-4283-adc6-99b133c49382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:05.612887 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259cb6fc-1844-4283-adc6-99b133c49382 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:05.613982 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259cb6fc-1844-4283-adc6-99b133c49382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:05.613982 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259cb6fc-1844-4283-adc6-99b133c49382 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:05.614923 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259cb6fc-1844-4283-adc6-99b133c49382 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:15:05.615437 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259cb6fc-1844-4283-adc6-99b133c49382 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:05.623774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259cb6fc-1844-4283-adc6-99b133c49382 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:05.653622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 908/1809] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:15:05.655434 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71ffec-4231-4ea5-94d1-98792ba55b36 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:05.680914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71ffec-4231-4ea5-94d1-98792ba55b36 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:05.691781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc597a-de82-42aa-ae53-20310779175f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:05.692824 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc597a-de82-42aa-ae53-20310779175f admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:05.693491 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc597a-de82-42aa-ae53-20310779175f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:05.694156 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc597a-de82-42aa-ae53-20310779175f admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:05.695261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc597a-de82-42aa-ae53-20310779175f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:05.700228 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc597a-de82-42aa-ae53-20310779175f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:05.892965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 902/1810] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:15:05 2022] POST /identity/v3/projects => generated 375 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:15:05.899400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259cb6fc-1844-4283-adc6-99b133c49382 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:05.917486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259cb6fc-1844-4283-adc6-99b133c49382 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:05.974657 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc597a-de82-42aa-ae53-20310779175f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:05.978387 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fabc597a-de82-42aa-ae53-20310779175f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:06.020544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486edb45-5a37-405b-9e4e-0b3cc8c56a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:06.022545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486edb45-5a37-405b-9e4e-0b3cc8c56a71 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:06.024431 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486edb45-5a37-405b-9e4e-0b3cc8c56a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:06.025679 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486edb45-5a37-405b-9e4e-0b3cc8c56a71 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:06.027003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486edb45-5a37-405b-9e4e-0b3cc8c56a71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:06.029476 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486edb45-5a37-405b-9e4e-0b3cc8c56a71 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:06.134414 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc597a-de82-42aa-ae53-20310779175f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:06.139580 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa967c24-7b1c-4e53-a166-1925562a903e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:06.140260 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa967c24-7b1c-4e53-a166-1925562a903e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:06.140582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa967c24-7b1c-4e53-a166-1925562a903e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:06.140918 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa967c24-7b1c-4e53-a166-1925562a903e admin admin] PATH_INFO: `/v3/users/cd6a46ec814c42cd904fc373039c2627` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:06.142243 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa967c24-7b1c-4e53-a166-1925562a903e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd6a46ec814c42cd904fc373039c2627)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:06.144294 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa967c24-7b1c-4e53-a166-1925562a903e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:06.164387 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:06.176230 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 909/1811] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:15:05 2022] POST /identity/v3/users => generated 288 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:15:06.182331 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486edb45-5a37-405b-9e4e-0b3cc8c56a71 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:06.192297 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486edb45-5a37-405b-9e4e-0b3cc8c56a71 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:06.199020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10be427-f5ff-4a1d-a92e-fea6609d3cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:06.199649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10be427-f5ff-4a1d-a92e-fea6609d3cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:06.199917 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10be427-f5ff-4a1d-a92e-fea6609d3cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:06.200187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10be427-f5ff-4a1d-a92e-fea6609d3cf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:06.200837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10be427-f5ff-4a1d-a92e-fea6609d3cf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:06.202460 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10be427-f5ff-4a1d-a92e-fea6609d3cf4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:06.216046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 910/1812] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:15:06.221523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10be427-f5ff-4a1d-a92e-fea6609d3cf4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:06.232029 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10be427-f5ff-4a1d-a92e-fea6609d3cf4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:06.236971 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ce6ff-99da-4d1e-9b9a-26c0007884dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:06.237608 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ce6ff-99da-4d1e-9b9a-26c0007884dd admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:06.237888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ce6ff-99da-4d1e-9b9a-26c0007884dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:06.238169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ce6ff-99da-4d1e-9b9a-26c0007884dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:06.238880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ce6ff-99da-4d1e-9b9a-26c0007884dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:06.240480 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ce6ff-99da-4d1e-9b9a-26c0007884dd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:06.251286 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 911/1813] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:15:06.255582 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ce6ff-99da-4d1e-9b9a-26c0007884dd admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:06.267258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ce6ff-99da-4d1e-9b9a-26c0007884dd admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:06.272999 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c2471-9949-4767-9ac4-d3873271dbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:06.273660 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c2471-9949-4767-9ac4-d3873271dbd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:06.273940 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c2471-9949-4767-9ac4-d3873271dbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:06.274224 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c2471-9949-4767-9ac4-d3873271dbd8 admin admin] PATH_INFO: `/v3/projects/72610255851d4b37add7611a10d1651f/users/cc8127689bdc47859ee6ffb8154e2ce6/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:06.288583 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:06.295408 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0c2471-9949-4767-9ac4-d3873271dbd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72610255851d4b37add7611a10d1651f, user_id=cc8127689bdc47859ee6ffb8154e2ce6, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:06.296871 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0c2471-9949-4767-9ac4-d3873271dbd8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:06.393097 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fa967c24-7b1c-4e53-a166-1925562a903e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd6a46ec814c42cd904fc373039c2627'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:06.460261 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 912/1814] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:15:06 2022] PUT /identity/v3/projects/72610255851d4b37add7611a10d1651f/users/cc8127689bdc47859ee6ffb8154e2ce6/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:15:06.477437 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fa967c24-7b1c-4e53-a166-1925562a903e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd6a46ec814c42cd904fc373039c2627'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:06.481220 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362aed18-d770-446c-9165-95af38fe6c6c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:06.481500 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362aed18-d770-446c-9165-95af38fe6c6c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:06.481994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362aed18-d770-446c-9165-95af38fe6c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:06.504558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fa967c24-7b1c-4e53-a166-1925562a903e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd6a46ec814c42cd904fc373039c2627'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:06.522752 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fa967c24-7b1c-4e53-a166-1925562a903e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd6a46ec814c42cd904fc373039c2627'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:06.523754 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fa967c24-7b1c-4e53-a166-1925562a903e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd6a46ec814c42cd904fc373039c2627'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:06.544937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-362aed18-d770-446c-9165-95af38fe6c6c None None] MFA Rules not processed for user `cc8127689bdc47859ee6ffb8154e2ce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:06.546548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-362aed18-d770-446c-9165-95af38fe6c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:06.556770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 903/1815] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:15:05 2022] DELETE /identity/v3/users/cd6a46ec814c42cd904fc373039c2627 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:06.564925 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e160bc0-5e76-498b-8b18-b70c492d1a76 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:06.565367 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e160bc0-5e76-498b-8b18-b70c492d1a76 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:06.565572 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e160bc0-5e76-498b-8b18-b70c492d1a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:06.618192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-9e160bc0-5e76-498b-8b18-b70c492d1a76 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:06.620017 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9e160bc0-5e76-498b-8b18-b70c492d1a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:06.727573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 913/1816] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:06.727693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 904/1817] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:06.740542 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362aed18-d770-446c-9165-95af38fe6c6c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:06.742808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e160bc0-5e76-498b-8b18-b70c492d1a76 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:06.744980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9e160bc0-5e76-498b-8b18-b70c492d1a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:06.745297 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-362aed18-d770-446c-9165-95af38fe6c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:06.827868 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e160bc0-5e76-498b-8b18-b70c492d1a76 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:06.833637 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b8f423-1c66-4c60-b88b-1e2615b17e52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:06.834492 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b8f423-1c66-4c60-b88b-1e2615b17e52 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:06.834873 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b8f423-1c66-4c60-b88b-1e2615b17e52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:06.835266 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b8f423-1c66-4c60-b88b-1e2615b17e52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:06.844837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362aed18-d770-446c-9165-95af38fe6c6c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:06.931426 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b8f423-1c66-4c60-b88b-1e2615b17e52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:06.937714 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b8f423-1c66-4c60-b88b-1e2615b17e52 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:06.959290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 905/1818] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:15:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:15:06.999976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c00cd5-e804-4fbc-9207-34c3355314f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:07.001025 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c00cd5-e804-4fbc-9207-34c3355314f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:07.001516 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c00cd5-e804-4fbc-9207-34c3355314f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:07.001999 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c00cd5-e804-4fbc-9207-34c3355314f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:07.032551 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c00cd5-e804-4fbc-9207-34c3355314f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:07.035196 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c00cd5-e804-4fbc-9207-34c3355314f0 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:07.059543 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 914/1819] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:15:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:15:07.344817 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:08.526842 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b8f423-1c66-4c60-b88b-1e2615b17e52 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:08.532075 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-58b8f423-1c66-4c60-b88b-1e2615b17e52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:08.557150 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b8f423-1c66-4c60-b88b-1e2615b17e52 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:08.579478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995db3e-72af-46ee-9ea0-9a0e14ef298a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:08.581454 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995db3e-72af-46ee-9ea0-9a0e14ef298a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:08.583528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995db3e-72af-46ee-9ea0-9a0e14ef298a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:08.584339 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995db3e-72af-46ee-9ea0-9a0e14ef298a admin admin] PATH_INFO: `/v3/projects/66236e5ec6ea4d4592b0af56771c5d0d` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:08.589210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8995db3e-72af-46ee-9ea0-9a0e14ef298a admin admin] RBAC: Authorizing `identity:delete_project(project_id=66236e5ec6ea4d4592b0af56771c5d0d)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:08.594820 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8995db3e-72af-46ee-9ea0-9a0e14ef298a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:08.800394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8995db3e-72af-46ee-9ea0-9a0e14ef298a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66236e5ec6ea4d4592b0af56771c5d0d', 'request_id': 'req-8995db3e-72af-46ee-9ea0-9a0e14ef298a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:08.892803 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8995db3e-72af-46ee-9ea0-9a0e14ef298a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66236e5ec6ea4d4592b0af56771c5d0d', 'request_id': 'req-8995db3e-72af-46ee-9ea0-9a0e14ef298a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:08.904896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8995db3e-72af-46ee-9ea0-9a0e14ef298a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66236e5ec6ea4d4592b0af56771c5d0d', 'request_id': 'req-8995db3e-72af-46ee-9ea0-9a0e14ef298a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:08.923315 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8995db3e-72af-46ee-9ea0-9a0e14ef298a admin admin] The token cache is being invalidate because project 66236e5ec6ea4d4592b0af56771c5d0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:15:08.923580 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8995db3e-72af-46ee-9ea0-9a0e14ef298a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:08.924540 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8995db3e-72af-46ee-9ea0-9a0e14ef298a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:08.926277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 906/1820] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:15:08 2022] DELETE /identity/v3/projects/66236e5ec6ea4d4592b0af56771c5d0d => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:08.932778 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c00cd5-e804-4fbc-9207-34c3355314f0 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:08.951326 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c00cd5-e804-4fbc-9207-34c3355314f0 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:09.068132 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:09.144054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3cb86-e827-4e66-bc91-8488fe35827e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:09.144927 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3cb86-e827-4e66-bc91-8488fe35827e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:09.145388 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3cb86-e827-4e66-bc91-8488fe35827e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:09.145822 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3cb86-e827-4e66-bc91-8488fe35827e admin admin] PATH_INFO: `/v3/users/83ea6f5e9ec348268cee23d79345e65e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:09.149318 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3cb86-e827-4e66-bc91-8488fe35827e admin admin] RBAC: Authorizing `identity:delete_user(user_id=83ea6f5e9ec348268cee23d79345e65e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:09.151975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3cb86-e827-4e66-bc91-8488fe35827e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:09.449198 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-20f3cb86-e827-4e66-bc91-8488fe35827e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83ea6f5e9ec348268cee23d79345e65e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:09.575299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-20f3cb86-e827-4e66-bc91-8488fe35827e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83ea6f5e9ec348268cee23d79345e65e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:09.600616 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-20f3cb86-e827-4e66-bc91-8488fe35827e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83ea6f5e9ec348268cee23d79345e65e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:09.656420 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-20f3cb86-e827-4e66-bc91-8488fe35827e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83ea6f5e9ec348268cee23d79345e65e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:09.658001 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-20f3cb86-e827-4e66-bc91-8488fe35827e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83ea6f5e9ec348268cee23d79345e65e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:09.678261 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 915/1821] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:15:08 2022] DELETE /identity/v3/users/83ea6f5e9ec348268cee23d79345e65e => generated 0 bytes in 748 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:11.546150 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995db3e-72af-46ee-9ea0-9a0e14ef298a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:11.550628 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8995db3e-72af-46ee-9ea0-9a0e14ef298a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:11.576687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995db3e-72af-46ee-9ea0-9a0e14ef298a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:11.689993 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8887079-f53e-4e7d-a28d-7aae09741299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:11.691006 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8887079-f53e-4e7d-a28d-7aae09741299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:11.691304 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8887079-f53e-4e7d-a28d-7aae09741299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:11.691582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8887079-f53e-4e7d-a28d-7aae09741299 admin admin] PATH_INFO: `/v3/projects/f9bbf53915f840049c3e306423921d63` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:11.693518 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8887079-f53e-4e7d-a28d-7aae09741299 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9bbf53915f840049c3e306423921d63)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:11.696962 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8887079-f53e-4e7d-a28d-7aae09741299 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:11.971441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a8887079-f53e-4e7d-a28d-7aae09741299 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9bbf53915f840049c3e306423921d63', 'request_id': 'req-a8887079-f53e-4e7d-a28d-7aae09741299'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:12.073484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f7be4-e421-4be6-8aac-c259d8a4f5a3 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:12.073484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f7be4-e421-4be6-8aac-c259d8a4f5a3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:12.073484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f7be4-e421-4be6-8aac-c259d8a4f5a3 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:12.077952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 916/1822] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 10 17:15:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 17:15:12.086680 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc42c19-b014-4c49-b3ff-3b3bf7f26747 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:12.086965 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc42c19-b014-4c49-b3ff-3b3bf7f26747 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:12.087236 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc42c19-b014-4c49-b3ff-3b3bf7f26747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:12.097973 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a8887079-f53e-4e7d-a28d-7aae09741299 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9bbf53915f840049c3e306423921d63', 'request_id': 'req-a8887079-f53e-4e7d-a28d-7aae09741299'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:12.115166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a8887079-f53e-4e7d-a28d-7aae09741299 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9bbf53915f840049c3e306423921d63', 'request_id': 'req-a8887079-f53e-4e7d-a28d-7aae09741299'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:12.146982 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a8887079-f53e-4e7d-a28d-7aae09741299 admin admin] The token cache is being invalidate because project f9bbf53915f840049c3e306423921d63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:15:12.147699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a8887079-f53e-4e7d-a28d-7aae09741299 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:12.148600 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a8887079-f53e-4e7d-a28d-7aae09741299 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:12.157052 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 907/1823] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:15:11 2022] DELETE /identity/v3/projects/f9bbf53915f840049c3e306423921d63 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:12.161522 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5b4e7-74d7-4c4f-b574-259565fb3f21 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:12.161955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5b4e7-74d7-4c4f-b574-259565fb3f21 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:12.162504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5b4e7-74d7-4c4f-b574-259565fb3f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:12.176015 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:12.184278 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8bc42c19-b014-4c49-b3ff-3b3bf7f26747 None None] MFA Rules not processed for user `615cc36e21cc493b823e6cee695ac308`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:12.188031 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8bc42c19-b014-4c49-b3ff-3b3bf7f26747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:12.200642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4bd5b4e7-74d7-4c4f-b574-259565fb3f21 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:12.202785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4bd5b4e7-74d7-4c4f-b574-259565fb3f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:12.376559 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 908/1824] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:12.378023 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a5daf-7c3e-4b3a-bc94-a0ce8d729f12 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:12.378311 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a5daf-7c3e-4b3a-bc94-a0ce8d729f12 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:12.378610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a5daf-7c3e-4b3a-bc94-a0ce8d729f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:12.418466 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 917/1825] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 10 17:15:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:15:12.421505 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-566a5daf-7c3e-4b3a-bc94-a0ce8d729f12 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:12.423396 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-566a5daf-7c3e-4b3a-bc94-a0ce8d729f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:12.453411 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 909/1826] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:15:12.454258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc42c19-b014-4c49-b3ff-3b3bf7f26747 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:12.513558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc42c19-b014-4c49-b3ff-3b3bf7f26747 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:12.543765 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801f0cc-279a-47aa-9049-6ef32bd2a103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:12.543765 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801f0cc-279a-47aa-9049-6ef32bd2a103 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:12.544194 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801f0cc-279a-47aa-9049-6ef32bd2a103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:12.544463 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801f0cc-279a-47aa-9049-6ef32bd2a103 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:12.550884 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c801f0cc-279a-47aa-9049-6ef32bd2a103 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:15:12.551417 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c801f0cc-279a-47aa-9049-6ef32bd2a103 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:12.555357 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c801f0cc-279a-47aa-9049-6ef32bd2a103 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:12.591514 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 918/1827] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:15:12.597490 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a5daf-7c3e-4b3a-bc94-a0ce8d729f12 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:12.619597 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a5daf-7c3e-4b3a-bc94-a0ce8d729f12 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:12.634235 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60204507-44d8-4f4a-ac03-63520a7e4d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:12.636492 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60204507-44d8-4f4a-ac03-63520a7e4d1c admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:12.637069 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60204507-44d8-4f4a-ac03-63520a7e4d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:12.637781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60204507-44d8-4f4a-ac03-63520a7e4d1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:12.638615 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60204507-44d8-4f4a-ac03-63520a7e4d1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:12.642103 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60204507-44d8-4f4a-ac03-63520a7e4d1c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:12.761767 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801f0cc-279a-47aa-9049-6ef32bd2a103 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:12.764497 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c801f0cc-279a-47aa-9049-6ef32bd2a103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:12.798156 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 910/1828] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:15:12 2022] POST /identity/v3/projects => generated 365 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:15:12.814404 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60204507-44d8-4f4a-ac03-63520a7e4d1c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:12.824536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801f0cc-279a-47aa-9049-6ef32bd2a103 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:12.953042 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:12.979261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60204507-44d8-4f4a-ac03-63520a7e4d1c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:12.985947 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33a44d-d9a7-431e-8091-b696ebfe3d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:12.986620 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33a44d-d9a7-431e-8091-b696ebfe3d87 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:12.986897 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33a44d-d9a7-431e-8091-b696ebfe3d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:12.987171 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33a44d-d9a7-431e-8091-b696ebfe3d87 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:12.987825 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc33a44d-d9a7-431e-8091-b696ebfe3d87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:12.990023 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc33a44d-d9a7-431e-8091-b696ebfe3d87 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:13.070316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c8ed8-3a0a-4755-a9e9-71c31a72a12f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:13.072415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c8ed8-3a0a-4755-a9e9-71c31a72a12f service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:13.073295 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c8ed8-3a0a-4755-a9e9-71c31a72a12f service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:13.074305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c8ed8-3a0a-4755-a9e9-71c31a72a12f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:13.108989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c8ed8-3a0a-4755-a9e9-71c31a72a12f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:13.113759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c8ed8-3a0a-4755-a9e9-71c31a72a12f service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:13.142656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 919/1829] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:15:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:15:13.175854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 911/1830] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:15:12 2022] POST /identity/v3/users => generated 283 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:15:13.178690 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c8ed8-3a0a-4755-a9e9-71c31a72a12f service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:13.207393 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c8ed8-3a0a-4755-a9e9-71c31a72a12f service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:13.226377 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84424dc5-5b6f-42bf-871c-8fa7c4cb3185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:13.228212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84424dc5-5b6f-42bf-871c-8fa7c4cb3185 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:13.229593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84424dc5-5b6f-42bf-871c-8fa7c4cb3185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:13.230160 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84424dc5-5b6f-42bf-871c-8fa7c4cb3185 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:13.231513 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84424dc5-5b6f-42bf-871c-8fa7c4cb3185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:13.234950 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84424dc5-5b6f-42bf-871c-8fa7c4cb3185 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:13.256366 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 920/1831] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:15:13.261323 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33a44d-d9a7-431e-8091-b696ebfe3d87 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:13.279694 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33a44d-d9a7-431e-8091-b696ebfe3d87 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:13.291782 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb8aa57-3ec5-4182-afa5-ea0c61a13141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:13.292414 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8aa57-3ec5-4182-afa5-ea0c61a13141 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:13.292688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8aa57-3ec5-4182-afa5-ea0c61a13141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:13.296504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8aa57-3ec5-4182-afa5-ea0c61a13141 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:13.296504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb8aa57-3ec5-4182-afa5-ea0c61a13141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:13.302579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb8aa57-3ec5-4182-afa5-ea0c61a13141 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:13.329424 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84424dc5-5b6f-42bf-871c-8fa7c4cb3185 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:13.340758 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-84424dc5-5b6f-42bf-871c-8fa7c4cb3185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:13.352868 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 912/1832] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:15:13.360092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb8aa57-3ec5-4182-afa5-ea0c61a13141 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:13.384564 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84424dc5-5b6f-42bf-871c-8fa7c4cb3185 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:13.394762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb8aa57-3ec5-4182-afa5-ea0c61a13141 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:13.396301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5be3fe-bc0e-4058-8040-eeb902ac24a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:13.397739 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5be3fe-bc0e-4058-8040-eeb902ac24a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:13.398574 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5be3fe-bc0e-4058-8040-eeb902ac24a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:13.399363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5be3fe-bc0e-4058-8040-eeb902ac24a7 admin admin] PATH_INFO: `/v3/users/cc8127689bdc47859ee6ffb8154e2ce6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:13.403219 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5be3fe-bc0e-4058-8040-eeb902ac24a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc8127689bdc47859ee6ffb8154e2ce6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:13.408539 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f59e2-8a09-4a90-b6f5-198b3973847b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:13.408952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5be3fe-bc0e-4058-8040-eeb902ac24a7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:13.409222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f59e2-8a09-4a90-b6f5-198b3973847b admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:13.409508 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f59e2-8a09-4a90-b6f5-198b3973847b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:13.409756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f59e2-8a09-4a90-b6f5-198b3973847b admin admin] PATH_INFO: `/v3/projects/89d64753f57840db93f49e1e26ab6004/users/ca9d1e6b58ea4f63bbfa0947b67d0e18/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:13.457512 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f59e2-8a09-4a90-b6f5-198b3973847b admin admin] RBAC: Authorizing `identity:create_grant(project_id=89d64753f57840db93f49e1e26ab6004, user_id=ca9d1e6b58ea4f63bbfa0947b67d0e18, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:13.460706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f59e2-8a09-4a90-b6f5-198b3973847b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:13.635263 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 913/1833] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:15:13 2022] PUT /identity/v3/projects/89d64753f57840db93f49e1e26ab6004/users/ca9d1e6b58ea4f63bbfa0947b67d0e18/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:13.654788 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca4e76-d285-4b14-bc05-0f48055b7e15 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:13.655486 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca4e76-d285-4b14-bc05-0f48055b7e15 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:13.656239 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca4e76-d285-4b14-bc05-0f48055b7e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:13.762139 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ccca4e76-d285-4b14-bc05-0f48055b7e15 None None] MFA Rules not processed for user `ca9d1e6b58ea4f63bbfa0947b67d0e18`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:13.764536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ccca4e76-d285-4b14-bc05-0f48055b7e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:13.810136 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf5be3fe-bc0e-4058-8040-eeb902ac24a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc8127689bdc47859ee6ffb8154e2ce6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:13.910328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf5be3fe-bc0e-4058-8040-eeb902ac24a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc8127689bdc47859ee6ffb8154e2ce6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:13.935156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf5be3fe-bc0e-4058-8040-eeb902ac24a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc8127689bdc47859ee6ffb8154e2ce6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:13.973263 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf5be3fe-bc0e-4058-8040-eeb902ac24a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc8127689bdc47859ee6ffb8154e2ce6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:13.973263 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf5be3fe-bc0e-4058-8040-eeb902ac24a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc8127689bdc47859ee6ffb8154e2ce6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:13.987788 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 921/1834] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:15:13 2022] DELETE /identity/v3/users/cc8127689bdc47859ee6ffb8154e2ce6 => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:13.992436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac204aeb-7f4e-45e3-8c1b-ec836e15374e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:13.992741 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac204aeb-7f4e-45e3-8c1b-ec836e15374e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:13.992995 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac204aeb-7f4e-45e3-8c1b-ec836e15374e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:14.020761 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ac204aeb-7f4e-45e3-8c1b-ec836e15374e None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:14.022409 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ac204aeb-7f4e-45e3-8c1b-ec836e15374e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:14.084700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:14.092822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 914/1835] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:14.098909 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca4e76-d285-4b14-bc05-0f48055b7e15 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:14.106019 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ccca4e76-d285-4b14-bc05-0f48055b7e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:14.249405 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 922/1836] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:15:14.293744 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac204aeb-7f4e-45e3-8c1b-ec836e15374e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:14.298185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ac204aeb-7f4e-45e3-8c1b-ec836e15374e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:14.363406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca4e76-d285-4b14-bc05-0f48055b7e15 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:14.375486 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0434ea-f4bb-4318-8b95-ee87b06f542c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:14.376463 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0434ea-f4bb-4318-8b95-ee87b06f542c admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:14.377257 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0434ea-f4bb-4318-8b95-ee87b06f542c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:14.377888 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0434ea-f4bb-4318-8b95-ee87b06f542c admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:14.379271 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0434ea-f4bb-4318-8b95-ee87b06f542c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:14.383727 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0434ea-f4bb-4318-8b95-ee87b06f542c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:14.462085 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac204aeb-7f4e-45e3-8c1b-ec836e15374e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:14.470133 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b270ee4-92ea-4c2f-afdd-9abb3556e7df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:14.471417 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b270ee4-92ea-4c2f-afdd-9abb3556e7df service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:14.471758 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b270ee4-92ea-4c2f-afdd-9abb3556e7df service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:14.472270 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b270ee4-92ea-4c2f-afdd-9abb3556e7df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:14.492839 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b270ee4-92ea-4c2f-afdd-9abb3556e7df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:14.499397 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b270ee4-92ea-4c2f-afdd-9abb3556e7df service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:14.532732 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 923/1837] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:15:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:15:14.558094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 915/1838] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:15:14 2022] POST /identity/v3/projects => generated 367 bytes in 459 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:15:14.565326 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b270ee4-92ea-4c2f-afdd-9abb3556e7df service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:14.593182 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b270ee4-92ea-4c2f-afdd-9abb3556e7df service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:14.735093 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308a068-bbec-40c4-bc16-3b7ed0a7cbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:14.736126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308a068-bbec-40c4-bc16-3b7ed0a7cbf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:14.736840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308a068-bbec-40c4-bc16-3b7ed0a7cbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:14.737322 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308a068-bbec-40c4-bc16-3b7ed0a7cbf2 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:14.738526 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7308a068-bbec-40c4-bc16-3b7ed0a7cbf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:14.742315 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7308a068-bbec-40c4-bc16-3b7ed0a7cbf2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:14.869837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 924/1839] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:15:14 2022] POST /identity/v3/users => generated 290 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:15:14.871742 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0434ea-f4bb-4318-8b95-ee87b06f542c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:14.886499 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0434ea-f4bb-4318-8b95-ee87b06f542c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:14.892923 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc11f4-42ee-458b-bebb-9dde3bbbe0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:14.893745 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc11f4-42ee-458b-bebb-9dde3bbbe0ae admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:14.894187 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc11f4-42ee-458b-bebb-9dde3bbbe0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:14.894673 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc11f4-42ee-458b-bebb-9dde3bbbe0ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:14.895660 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cc11f4-42ee-458b-bebb-9dde3bbbe0ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:14.897553 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cc11f4-42ee-458b-bebb-9dde3bbbe0ae admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:14.916282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 916/1840] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:15:14.923109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308a068-bbec-40c4-bc16-3b7ed0a7cbf2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:14.942269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308a068-bbec-40c4-bc16-3b7ed0a7cbf2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:14.950331 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9382bff-c7bc-4106-a37c-cc86f156e814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:14.951200 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9382bff-c7bc-4106-a37c-cc86f156e814 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:14.951372 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9382bff-c7bc-4106-a37c-cc86f156e814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:14.951628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9382bff-c7bc-4106-a37c-cc86f156e814 admin admin] PATH_INFO: `/v3/projects/7fc28a3214084728bbfb0d0fbc5a2902/users/5fb224e1b40342258c24a747e965042d/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:14.974715 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9382bff-c7bc-4106-a37c-cc86f156e814 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fc28a3214084728bbfb0d0fbc5a2902, user_id=5fb224e1b40342258c24a747e965042d, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:14.983743 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9382bff-c7bc-4106-a37c-cc86f156e814 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:15.100070 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:15.109248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 925/1841] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:15:14 2022] PUT /identity/v3/projects/7fc28a3214084728bbfb0d0fbc5a2902/users/5fb224e1b40342258c24a747e965042d/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:15.113690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc11f4-42ee-458b-bebb-9dde3bbbe0ae admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:15.138475 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc11f4-42ee-458b-bebb-9dde3bbbe0ae admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:15.216567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effdd870-7b1f-4a20-9885-cc19371025de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:15.217464 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effdd870-7b1f-4a20-9885-cc19371025de admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:15.217915 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effdd870-7b1f-4a20-9885-cc19371025de admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:15.218448 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effdd870-7b1f-4a20-9885-cc19371025de admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:15.219662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effdd870-7b1f-4a20-9885-cc19371025de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:15.222869 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effdd870-7b1f-4a20-9885-cc19371025de admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:15.235944 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 917/1842] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:15:15.241336 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9382bff-c7bc-4106-a37c-cc86f156e814 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:15.260586 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9382bff-c7bc-4106-a37c-cc86f156e814 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:15.268823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf4324-0d3c-454b-bd11-df38704b7571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:15.269627 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf4324-0d3c-454b-bd11-df38704b7571 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:15.270058 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf4324-0d3c-454b-bd11-df38704b7571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:15.270437 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf4324-0d3c-454b-bd11-df38704b7571 admin admin] PATH_INFO: `/v3/projects/7fc28a3214084728bbfb0d0fbc5a2902/users/5fb224e1b40342258c24a747e965042d/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:15.273093 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf4324-0d3c-454b-bd11-df38704b7571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fc28a3214084728bbfb0d0fbc5a2902, user_id=5fb224e1b40342258c24a747e965042d, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:15.277740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf4324-0d3c-454b-bd11-df38704b7571 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:15.291590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 926/1843] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:15:15 2022] PUT /identity/v3/projects/7fc28a3214084728bbfb0d0fbc5a2902/users/5fb224e1b40342258c24a747e965042d/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:15.302383 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529988c-623c-4374-a5d5-386263484c37 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:15.302663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529988c-623c-4374-a5d5-386263484c37 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:15.302996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529988c-623c-4374-a5d5-386263484c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:15.338946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2529988c-623c-4374-a5d5-386263484c37 None None] MFA Rules not processed for user `5fb224e1b40342258c24a747e965042d`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:15.340003 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2529988c-623c-4374-a5d5-386263484c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:15.340120 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf4324-0d3c-454b-bd11-df38704b7571 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:15.341942 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-45cf4324-0d3c-454b-bd11-df38704b7571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:15.483164 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf4324-0d3c-454b-bd11-df38704b7571 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:15.490111 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d0375-1981-4659-a22e-e0ac332411a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:15.490716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 918/1844] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:15 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:15.505119 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d0375-1981-4659-a22e-e0ac332411a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:15.505908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d0375-1981-4659-a22e-e0ac332411a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:15.506724 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d0375-1981-4659-a22e-e0ac332411a4 admin admin] PATH_INFO: `/v3/projects/72610255851d4b37add7611a10d1651f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:15.508846 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d0375-1981-4659-a22e-e0ac332411a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72610255851d4b37add7611a10d1651f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:15.514318 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d0375-1981-4659-a22e-e0ac332411a4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:15.516293 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529988c-623c-4374-a5d5-386263484c37 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:15.523877 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2529988c-623c-4374-a5d5-386263484c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:15.565770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:15.692422 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529988c-623c-4374-a5d5-386263484c37 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:15.696712 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b7690-5bbf-4d10-8cbb-a66fdff0ffb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:15.698341 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b7690-5bbf-4d10-8cbb-a66fdff0ffb8 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:15.698904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b7690-5bbf-4d10-8cbb-a66fdff0ffb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:15.699421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b7690-5bbf-4d10-8cbb-a66fdff0ffb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:15.713444 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b7690-5bbf-4d10-8cbb-a66fdff0ffb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:15.715554 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b7690-5bbf-4d10-8cbb-a66fdff0ffb8 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:15.727392 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 919/1845] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:15:15 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:15:15.774574 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a95d0375-1981-4659-a22e-e0ac332411a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72610255851d4b37add7611a10d1651f', 'request_id': 'req-a95d0375-1981-4659-a22e-e0ac332411a4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:15.789132 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a95d0375-1981-4659-a22e-e0ac332411a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72610255851d4b37add7611a10d1651f', 'request_id': 'req-a95d0375-1981-4659-a22e-e0ac332411a4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:15.900883 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a95d0375-1981-4659-a22e-e0ac332411a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72610255851d4b37add7611a10d1651f', 'request_id': 'req-a95d0375-1981-4659-a22e-e0ac332411a4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:15.932098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a95d0375-1981-4659-a22e-e0ac332411a4 admin admin] The token cache is being invalidate because project 72610255851d4b37add7611a10d1651f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:15:15.932584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a95d0375-1981-4659-a22e-e0ac332411a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:15.933902 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a95d0375-1981-4659-a22e-e0ac332411a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:15.937060 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 927/1846] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:15:15 2022] DELETE /identity/v3/projects/72610255851d4b37add7611a10d1651f => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:15.941792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff293792-a483-4258-ae21-f1a507788a00 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:15.942422 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff293792-a483-4258-ae21-f1a507788a00 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:15.944069 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff293792-a483-4258-ae21-f1a507788a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:15.969573 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ff293792-a483-4258-ae21-f1a507788a00 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:15.971089 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ff293792-a483-4258-ae21-f1a507788a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:16.054653 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 920/1847] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:16.061992 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34ea05-ca11-479f-ad1d-649e83d8b2df None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:16.062516 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34ea05-ca11-479f-ad1d-649e83d8b2df None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:16.062944 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34ea05-ca11-479f-ad1d-649e83d8b2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:16.085101 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4b34ea05-ca11-479f-ad1d-649e83d8b2df None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:16.086634 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4b34ea05-ca11-479f-ad1d-649e83d8b2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:16.095671 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 928/1848] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:16.100513 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff293792-a483-4258-ae21-f1a507788a00 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:16.112531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff293792-a483-4258-ae21-f1a507788a00 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:16.168335 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7afcec-9622-4c81-82e6-8d30329e936b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:16.169368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7afcec-9622-4c81-82e6-8d30329e936b admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:16.169720 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7afcec-9622-4c81-82e6-8d30329e936b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:16.170029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7afcec-9622-4c81-82e6-8d30329e936b admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:16.171165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7afcec-9622-4c81-82e6-8d30329e936b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:15:16.172280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34ea05-ca11-479f-ad1d-649e83d8b2df None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:16.174086 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4b34ea05-ca11-479f-ad1d-649e83d8b2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:16.182477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7afcec-9622-4c81-82e6-8d30329e936b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:16.191326 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7afcec-9622-4c81-82e6-8d30329e936b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:16.233394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 921/1849] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:15:16.234780 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7afcec-9622-4c81-82e6-8d30329e936b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:16.256542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7afcec-9622-4c81-82e6-8d30329e936b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:16.258652 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34ea05-ca11-479f-ad1d-649e83d8b2df None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:16.264638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b4605-d6d2-4344-9fcc-8a8768931aba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:16.265750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09694d80-ffcd-4945-bbb8-34ca79474adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:16.266307 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b4605-d6d2-4344-9fcc-8a8768931aba service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:16.266474 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09694d80-ffcd-4945-bbb8-34ca79474adf admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:16.266661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09694d80-ffcd-4945-bbb8-34ca79474adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:16.266914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09694d80-ffcd-4945-bbb8-34ca79474adf admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:16.267527 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b4605-d6d2-4344-9fcc-8a8768931aba service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:16.267636 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09694d80-ffcd-4945-bbb8-34ca79474adf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:16.268178 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b4605-d6d2-4344-9fcc-8a8768931aba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:16.269551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09694d80-ffcd-4945-bbb8-34ca79474adf admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:16.273392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7b0b4605-d6d2-4344-9fcc-8a8768931aba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:16.380099 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 922/1850] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:15:16 2022] POST /identity/v3/projects => generated 381 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:15:16.385275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09694d80-ffcd-4945-bbb8-34ca79474adf admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:16.393216 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b4605-d6d2-4344-9fcc-8a8768931aba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:16.394714 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b4605-d6d2-4344-9fcc-8a8768931aba service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:16.404725 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 929/1851] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:15:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:15:16.469439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09694d80-ffcd-4945-bbb8-34ca79474adf admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:16.474649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb7053-2e50-4eeb-b7b2-203a283bf5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:16.475275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb7053-2e50-4eeb-b7b2-203a283bf5df admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:16.475547 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb7053-2e50-4eeb-b7b2-203a283bf5df admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:16.475813 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb7053-2e50-4eeb-b7b2-203a283bf5df admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:16.476432 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb7053-2e50-4eeb-b7b2-203a283bf5df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:16.478987 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb7053-2e50-4eeb-b7b2-203a283bf5df admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:16.575520 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 923/1852] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:15:16 2022] POST /identity/v3/users => generated 291 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:15:16.579959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b4605-d6d2-4344-9fcc-8a8768931aba service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:16.592150 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b4605-d6d2-4344-9fcc-8a8768931aba service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:16.608930 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4247e-3564-477e-8200-87499c65363e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:16.608930 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4247e-3564-477e-8200-87499c65363e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:16.608930 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4247e-3564-477e-8200-87499c65363e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:16.608930 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4247e-3564-477e-8200-87499c65363e admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:16.609291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4247e-3564-477e-8200-87499c65363e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:16.609291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4247e-3564-477e-8200-87499c65363e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:16.619039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 930/1853] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:15:16.622836 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb7053-2e50-4eeb-b7b2-203a283bf5df admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:16.636741 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb7053-2e50-4eeb-b7b2-203a283bf5df admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:16.644875 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06bf36-e66b-4964-aecf-5710b3144e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:16.645546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06bf36-e66b-4964-aecf-5710b3144e8e admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:16.646018 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06bf36-e66b-4964-aecf-5710b3144e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:16.646525 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06bf36-e66b-4964-aecf-5710b3144e8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:16.647690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06bf36-e66b-4964-aecf-5710b3144e8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:16.651441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06bf36-e66b-4964-aecf-5710b3144e8e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:16.674790 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 924/1854] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:15:16.678666 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4247e-3564-477e-8200-87499c65363e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:16.691894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4247e-3564-477e-8200-87499c65363e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:16.697429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb807b5-a9ae-4ca1-a526-a4a5d6f19694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:16.699488 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb807b5-a9ae-4ca1-a526-a4a5d6f19694 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:16.699488 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb807b5-a9ae-4ca1-a526-a4a5d6f19694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:16.699488 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb807b5-a9ae-4ca1-a526-a4a5d6f19694 admin admin] PATH_INFO: `/v3/projects/7216835c713b4b45b0c060b32e44608d/users/4df400ec0c844bfdbad7589f405fd225/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:16.718199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb807b5-a9ae-4ca1-a526-a4a5d6f19694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7216835c713b4b45b0c060b32e44608d, user_id=4df400ec0c844bfdbad7589f405fd225, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:16.721041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb807b5-a9ae-4ca1-a526-a4a5d6f19694 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:16.792350 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 931/1855] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:15:16 2022] PUT /identity/v3/projects/7216835c713b4b45b0c060b32e44608d/users/4df400ec0c844bfdbad7589f405fd225/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:16.807293 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1928edc-8442-4f22-92d0-5545131b4350 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:16.807499 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1928edc-8442-4f22-92d0-5545131b4350 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:16.807929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1928edc-8442-4f22-92d0-5545131b4350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:16.842183 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a1928edc-8442-4f22-92d0-5545131b4350 None None] MFA Rules not processed for user `4df400ec0c844bfdbad7589f405fd225`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:16.843501 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a1928edc-8442-4f22-92d0-5545131b4350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:16.879296 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:16.952683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb807b5-a9ae-4ca1-a526-a4a5d6f19694 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:16.962127 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 925/1856] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:17.057575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb807b5-a9ae-4ca1-a526-a4a5d6f19694 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:17.062644 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e893a6-0421-436c-bbc0-7cbae925ed00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:17.063388 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e893a6-0421-436c-bbc0-7cbae925ed00 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:17.063814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e893a6-0421-436c-bbc0-7cbae925ed00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:17.064092 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e893a6-0421-436c-bbc0-7cbae925ed00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:17.064748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e893a6-0421-436c-bbc0-7cbae925ed00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:17.066829 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e893a6-0421-436c-bbc0-7cbae925ed00 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:17.123820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:17.241006 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 932/1857] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:15:16 2022] POST /identity/v3/projects => generated 385 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:15:17.275026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1928edc-8442-4f22-92d0-5545131b4350 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:17.275026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1928edc-8442-4f22-92d0-5545131b4350 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:17.327895 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54868ab6-6441-48bd-b498-4aaeed48a7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:17.328501 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54868ab6-6441-48bd-b498-4aaeed48a7da admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:17.328776 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54868ab6-6441-48bd-b498-4aaeed48a7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:17.329044 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54868ab6-6441-48bd-b498-4aaeed48a7da admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:17.329669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54868ab6-6441-48bd-b498-4aaeed48a7da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:17.331728 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54868ab6-6441-48bd-b498-4aaeed48a7da admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:17.468687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 926/1858] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:15:17 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:15:17.472575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e893a6-0421-436c-bbc0-7cbae925ed00 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:17.486544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e893a6-0421-436c-bbc0-7cbae925ed00 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:17.503444 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f05c13-2cf3-4e98-b90b-bef946a3401c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:17.504778 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f05c13-2cf3-4e98-b90b-bef946a3401c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:17.505574 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f05c13-2cf3-4e98-b90b-bef946a3401c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:17.506513 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f05c13-2cf3-4e98-b90b-bef946a3401c admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:17.508087 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f05c13-2cf3-4e98-b90b-bef946a3401c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:17.514008 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f05c13-2cf3-4e98-b90b-bef946a3401c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:17.549872 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 933/1859] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:15:17.550715 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54868ab6-6441-48bd-b498-4aaeed48a7da admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:17.580591 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54868ab6-6441-48bd-b498-4aaeed48a7da admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:17.583277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc1a78-1021-4092-af35-64baaff80f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:17.584285 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc1a78-1021-4092-af35-64baaff80f11 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:17.584749 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc1a78-1021-4092-af35-64baaff80f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:17.585215 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc1a78-1021-4092-af35-64baaff80f11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:17.586506 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc1a78-1021-4092-af35-64baaff80f11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:17.590642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc1a78-1021-4092-af35-64baaff80f11 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:17.616874 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 927/1860] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:15:17.621154 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f05c13-2cf3-4e98-b90b-bef946a3401c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:17.633481 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f05c13-2cf3-4e98-b90b-bef946a3401c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:17.641599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f08e5-16b8-447e-9771-f21c79dde732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:17.646458 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f08e5-16b8-447e-9771-f21c79dde732 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:17.646458 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f08e5-16b8-447e-9771-f21c79dde732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:17.646458 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f08e5-16b8-447e-9771-f21c79dde732 admin admin] PATH_INFO: `/v3/projects/8e16765cf8b44c66823a4a4cf9862b99/users/e1721f8e3adf4c499af2f27577114cf3/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:17.665049 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f08e5-16b8-447e-9771-f21c79dde732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e16765cf8b44c66823a4a4cf9862b99, user_id=e1721f8e3adf4c499af2f27577114cf3, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:17.669443 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f08e5-16b8-447e-9771-f21c79dde732 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:17.798383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 934/1861] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:15:17 2022] PUT /identity/v3/projects/8e16765cf8b44c66823a4a4cf9862b99/users/e1721f8e3adf4c499af2f27577114cf3/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:17.805125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf2b31-d5f3-4130-aecb-03f124fe4781 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:17.805706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf2b31-d5f3-4130-aecb-03f124fe4781 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:17.807557 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf2b31-d5f3-4130-aecb-03f124fe4781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:17.865811 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2adf2b31-d5f3-4130-aecb-03f124fe4781 None None] MFA Rules not processed for user `e1721f8e3adf4c499af2f27577114cf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:17.867999 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2adf2b31-d5f3-4130-aecb-03f124fe4781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:17.966073 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 928/1862] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:15:17.966997 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f08e5-16b8-447e-9771-f21c79dde732 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:18.055680 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:18.070298 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f08e5-16b8-447e-9771-f21c79dde732 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:18.079228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42406b-8869-405d-b464-e3f2591155e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:18.080284 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42406b-8869-405d-b464-e3f2591155e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:18.080781 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42406b-8869-405d-b464-e3f2591155e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:18.081265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42406b-8869-405d-b464-e3f2591155e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:18.082484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42406b-8869-405d-b464-e3f2591155e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:18.085991 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42406b-8869-405d-b464-e3f2591155e9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:18.195806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 935/1863] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:15:17 2022] POST /identity/v3/projects => generated 385 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:15:18.201081 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf2b31-d5f3-4130-aecb-03f124fe4781 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:18.212248 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf2b31-d5f3-4130-aecb-03f124fe4781 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:18.218313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:18.302921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528fda5e-59f9-42b7-99c0-1261deaab627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:18.304300 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fda5e-59f9-42b7-99c0-1261deaab627 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:18.304780 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fda5e-59f9-42b7-99c0-1261deaab627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:18.305242 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fda5e-59f9-42b7-99c0-1261deaab627 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:18.307145 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528fda5e-59f9-42b7-99c0-1261deaab627 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:18.310887 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528fda5e-59f9-42b7-99c0-1261deaab627 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:18.424940 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 929/1864] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:15:18 2022] POST /identity/v3/users => generated 299 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:15:18.429277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42406b-8869-405d-b464-e3f2591155e9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:18.442537 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42406b-8869-405d-b464-e3f2591155e9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:18.449470 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006aaeeb-0ff9-49a2-bf6c-2a07a78fed7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:18.450768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006aaeeb-0ff9-49a2-bf6c-2a07a78fed7c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:18.451263 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006aaeeb-0ff9-49a2-bf6c-2a07a78fed7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:18.451746 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006aaeeb-0ff9-49a2-bf6c-2a07a78fed7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:18.453205 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006aaeeb-0ff9-49a2-bf6c-2a07a78fed7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:18.456335 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006aaeeb-0ff9-49a2-bf6c-2a07a78fed7c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:18.477357 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 936/1865] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:15:18.482971 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528fda5e-59f9-42b7-99c0-1261deaab627 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:18.498317 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528fda5e-59f9-42b7-99c0-1261deaab627 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:18.503806 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b11de-fc2b-46c5-9e79-6ea22bcba848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:18.504274 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b11de-fc2b-46c5-9e79-6ea22bcba848 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:18.505141 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b11de-fc2b-46c5-9e79-6ea22bcba848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:18.505141 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b11de-fc2b-46c5-9e79-6ea22bcba848 admin admin] PATH_INFO: `/v3/projects/a575455a3d2f4277bae898425ebbee06/users/be4b056a23a74e558709c86ad9e7abd6/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:18.531990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b11de-fc2b-46c5-9e79-6ea22bcba848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a575455a3d2f4277bae898425ebbee06, user_id=be4b056a23a74e558709c86ad9e7abd6, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:18.537841 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b11de-fc2b-46c5-9e79-6ea22bcba848 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:18.637649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 930/1866] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:15:18 2022] PUT /identity/v3/projects/a575455a3d2f4277bae898425ebbee06/users/be4b056a23a74e558709c86ad9e7abd6/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:15:18.642947 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006aaeeb-0ff9-49a2-bf6c-2a07a78fed7c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:18.656658 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006aaeeb-0ff9-49a2-bf6c-2a07a78fed7c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:18.737135 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e064d-bbfa-42e4-a24d-bfd327562e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:18.738168 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e064d-bbfa-42e4-a24d-bfd327562e92 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:18.738660 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e064d-bbfa-42e4-a24d-bfd327562e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:18.739104 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e064d-bbfa-42e4-a24d-bfd327562e92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:18.740126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e064d-bbfa-42e4-a24d-bfd327562e92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:18.742852 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e064d-bbfa-42e4-a24d-bfd327562e92 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:18.760195 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 937/1867] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:15:18.766529 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b11de-fc2b-46c5-9e79-6ea22bcba848 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:18.780404 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b11de-fc2b-46c5-9e79-6ea22bcba848 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:18.787474 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe657ca-1752-4432-8adf-54f6f94cd917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:18.788279 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe657ca-1752-4432-8adf-54f6f94cd917 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:18.788701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe657ca-1752-4432-8adf-54f6f94cd917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:18.789164 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe657ca-1752-4432-8adf-54f6f94cd917 admin admin] PATH_INFO: `/v3/projects/a575455a3d2f4277bae898425ebbee06/users/be4b056a23a74e558709c86ad9e7abd6/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:18.792927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe657ca-1752-4432-8adf-54f6f94cd917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a575455a3d2f4277bae898425ebbee06, user_id=be4b056a23a74e558709c86ad9e7abd6, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:18.797072 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe657ca-1752-4432-8adf-54f6f94cd917 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:18.811928 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 931/1868] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:15:18 2022] PUT /identity/v3/projects/a575455a3d2f4277bae898425ebbee06/users/be4b056a23a74e558709c86ad9e7abd6/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:15:18.823587 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e717f6-edf1-4bf5-afa5-fd738a3c9e8d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:18.823877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e717f6-edf1-4bf5-afa5-fd738a3c9e8d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:18.824152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e717f6-edf1-4bf5-afa5-fd738a3c9e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:18.857631 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-08e717f6-edf1-4bf5-afa5-fd738a3c9e8d None None] MFA Rules not processed for user `be4b056a23a74e558709c86ad9e7abd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:18.859138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-08e717f6-edf1-4bf5-afa5-fd738a3c9e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:18.961894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 938/1869] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:18 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:15:18.979268 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe657ca-1752-4432-8adf-54f6f94cd917 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:18.981514 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-afe657ca-1752-4432-8adf-54f6f94cd917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:19.089620 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe657ca-1752-4432-8adf-54f6f94cd917 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:19.095732 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327459a1-0b06-4b94-b186-7a93abf8d23f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:19.096850 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327459a1-0b06-4b94-b186-7a93abf8d23f service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:19.097423 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327459a1-0b06-4b94-b186-7a93abf8d23f service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:19.097989 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327459a1-0b06-4b94-b186-7a93abf8d23f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:19.106203 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:19.188309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327459a1-0b06-4b94-b186-7a93abf8d23f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:19.189750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327459a1-0b06-4b94-b186-7a93abf8d23f service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:19.201385 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 932/1870] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:15:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:15:19.489948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:21.579764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e717f6-edf1-4bf5-afa5-fd738a3c9e8d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:21.597162 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e717f6-edf1-4bf5-afa5-fd738a3c9e8d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:21.602342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c1d96-2a68-4889-9475-1ce6706fa817 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:21.604837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c1d96-2a68-4889-9475-1ce6706fa817 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:21.605255 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c1d96-2a68-4889-9475-1ce6706fa817 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:21.605843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c1d96-2a68-4889-9475-1ce6706fa817 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:21.739392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c1d96-2a68-4889-9475-1ce6706fa817 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:21.742177 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c1d96-2a68-4889-9475-1ce6706fa817 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:21.761171 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 939/1871] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:15:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:15:24.395399 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327459a1-0b06-4b94-b186-7a93abf8d23f service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:24.408490 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327459a1-0b06-4b94-b186-7a93abf8d23f service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:24.414084 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13511e47-d1fc-42b8-adea-145afbd50aee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:24.416536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13511e47-d1fc-42b8-adea-145afbd50aee service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:24.417049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13511e47-d1fc-42b8-adea-145afbd50aee service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:24.417545 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13511e47-d1fc-42b8-adea-145afbd50aee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:24.437395 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13511e47-d1fc-42b8-adea-145afbd50aee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:24.440160 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13511e47-d1fc-42b8-adea-145afbd50aee service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:24.461819 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 933/1872] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:15:24 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:15:27.276797 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c1d96-2a68-4889-9475-1ce6706fa817 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:27.278914 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cf0c1d96-2a68-4889-9475-1ce6706fa817 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:27.408793 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c1d96-2a68-4889-9475-1ce6706fa817 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:27.418728 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca58349d-8d94-43ca-a296-5dd01af5eccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:27.420181 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58349d-8d94-43ca-a296-5dd01af5eccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:27.420646 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58349d-8d94-43ca-a296-5dd01af5eccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:27.421517 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58349d-8d94-43ca-a296-5dd01af5eccc admin admin] PATH_INFO: `/v3/users/ca9d1e6b58ea4f63bbfa0947b67d0e18` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:27.423545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca58349d-8d94-43ca-a296-5dd01af5eccc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca9d1e6b58ea4f63bbfa0947b67d0e18)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:27.426760 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca58349d-8d94-43ca-a296-5dd01af5eccc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:27.434936 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:27.678505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ca58349d-8d94-43ca-a296-5dd01af5eccc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca9d1e6b58ea4f63bbfa0947b67d0e18'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:27.767228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ca58349d-8d94-43ca-a296-5dd01af5eccc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca9d1e6b58ea4f63bbfa0947b67d0e18'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:27.783248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ca58349d-8d94-43ca-a296-5dd01af5eccc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca9d1e6b58ea4f63bbfa0947b67d0e18'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:27.805147 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ca58349d-8d94-43ca-a296-5dd01af5eccc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca9d1e6b58ea4f63bbfa0947b67d0e18'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:27.805909 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ca58349d-8d94-43ca-a296-5dd01af5eccc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca9d1e6b58ea4f63bbfa0947b67d0e18'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:27.813209 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 940/1873] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:15:27 2022] DELETE /identity/v3/users/ca9d1e6b58ea4f63bbfa0947b67d0e18 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:27.820197 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795781d4-15fd-400d-9ba8-c0cb6435eeeb None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:27.820719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795781d4-15fd-400d-9ba8-c0cb6435eeeb None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:27.821207 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795781d4-15fd-400d-9ba8-c0cb6435eeeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:27.840752 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca58349d-8d94-43ca-a296-5dd01af5eccc admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:27.842802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ca58349d-8d94-43ca-a296-5dd01af5eccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:27.851937 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-795781d4-15fd-400d-9ba8-c0cb6435eeeb None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:27.853488 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-795781d4-15fd-400d-9ba8-c0cb6435eeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:27.972167 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 934/1874] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:27.996916 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795781d4-15fd-400d-9ba8-c0cb6435eeeb None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:27.998667 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-795781d4-15fd-400d-9ba8-c0cb6435eeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:28.049259 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca58349d-8d94-43ca-a296-5dd01af5eccc admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:28.059878 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa179d-23d9-4088-b922-2b3797297e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:28.059878 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa179d-23d9-4088-b922-2b3797297e73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:28.059878 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa179d-23d9-4088-b922-2b3797297e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:28.059878 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa179d-23d9-4088-b922-2b3797297e73 admin admin] PATH_INFO: `/v3/users/e7e4aad5aef24a7f8d163a8b07c51e83` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:28.063205 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa179d-23d9-4088-b922-2b3797297e73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7e4aad5aef24a7f8d163a8b07c51e83)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:28.076427 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa179d-23d9-4088-b922-2b3797297e73 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:28.109712 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795781d4-15fd-400d-9ba8-c0cb6435eeeb None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:28.118258 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a401746-08dd-4bbf-ad0b-c3fde8cda17b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:28.121955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a401746-08dd-4bbf-ad0b-c3fde8cda17b service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:28.123515 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a401746-08dd-4bbf-ad0b-c3fde8cda17b service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:28.124133 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a401746-08dd-4bbf-ad0b-c3fde8cda17b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:28.141089 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a401746-08dd-4bbf-ad0b-c3fde8cda17b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:28.143026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a401746-08dd-4bbf-ad0b-c3fde8cda17b service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:28.166781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 935/1875] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:15:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:15:28.331616 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-19aa179d-23d9-4088-b922-2b3797297e73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7e4aad5aef24a7f8d163a8b07c51e83'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:28.469095 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-19aa179d-23d9-4088-b922-2b3797297e73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7e4aad5aef24a7f8d163a8b07c51e83'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:28.484275 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-19aa179d-23d9-4088-b922-2b3797297e73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7e4aad5aef24a7f8d163a8b07c51e83'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:28.527267 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-19aa179d-23d9-4088-b922-2b3797297e73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7e4aad5aef24a7f8d163a8b07c51e83'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:28.530457 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-19aa179d-23d9-4088-b922-2b3797297e73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7e4aad5aef24a7f8d163a8b07c51e83'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:28.551503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 941/1876] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:15:27 2022] DELETE /identity/v3/users/e7e4aad5aef24a7f8d163a8b07c51e83 => generated 0 bytes in 711 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:15:28.556572 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb18ec-451c-4715-a004-c377a54ecea3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:28.556836 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb18ec-451c-4715-a004-c377a54ecea3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:28.557110 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb18ec-451c-4715-a004-c377a54ecea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:28.605669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8abb18ec-451c-4715-a004-c377a54ecea3 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:28.606970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8abb18ec-451c-4715-a004-c377a54ecea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:28.656521 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:28.715528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 936/1877] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:28.737459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa179d-23d9-4088-b922-2b3797297e73 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:28.739971 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-19aa179d-23d9-4088-b922-2b3797297e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:28.766488 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:28.871668 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa179d-23d9-4088-b922-2b3797297e73 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:28.876565 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7d1c2-013b-439b-bd18-1628296ee99e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:28.877487 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7d1c2-013b-439b-bd18-1628296ee99e service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:28.877944 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7d1c2-013b-439b-bd18-1628296ee99e service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:28.878411 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7d1c2-013b-439b-bd18-1628296ee99e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:28.890944 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac7d1c2-013b-439b-bd18-1628296ee99e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:28.892783 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac7d1c2-013b-439b-bd18-1628296ee99e service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:28.923483 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 942/1878] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:15:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:15:28.956305 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb18ec-451c-4715-a004-c377a54ecea3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:28.975769 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8abb18ec-451c-4715-a004-c377a54ecea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:29.018595 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb18ec-451c-4715-a004-c377a54ecea3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:29.054879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e0279-55fe-43f9-8042-8894f119db94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:29.054879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e0279-55fe-43f9-8042-8894f119db94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:29.054879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e0279-55fe-43f9-8042-8894f119db94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:29.054879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e0279-55fe-43f9-8042-8894f119db94 admin admin] PATH_INFO: `/v3/projects/89d64753f57840db93f49e1e26ab6004` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:29.055750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e0279-55fe-43f9-8042-8894f119db94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89d64753f57840db93f49e1e26ab6004)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:29.055750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e0279-55fe-43f9-8042-8894f119db94 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:29.256792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-454e0279-55fe-43f9-8042-8894f119db94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89d64753f57840db93f49e1e26ab6004', 'request_id': 'req-454e0279-55fe-43f9-8042-8894f119db94'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:29.383475 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-454e0279-55fe-43f9-8042-8894f119db94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89d64753f57840db93f49e1e26ab6004', 'request_id': 'req-454e0279-55fe-43f9-8042-8894f119db94'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:29.407478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-454e0279-55fe-43f9-8042-8894f119db94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89d64753f57840db93f49e1e26ab6004', 'request_id': 'req-454e0279-55fe-43f9-8042-8894f119db94'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:29.460866 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-454e0279-55fe-43f9-8042-8894f119db94 admin admin] The token cache is being invalidate because project 89d64753f57840db93f49e1e26ab6004 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:15:29.461387 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-454e0279-55fe-43f9-8042-8894f119db94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:29.468516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-454e0279-55fe-43f9-8042-8894f119db94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:29.479698 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7d1c2-013b-439b-bd18-1628296ee99e service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:29.488334 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 937/1879] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:15:28 2022] DELETE /identity/v3/projects/89d64753f57840db93f49e1e26ab6004 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:29.516553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7d1c2-013b-439b-bd18-1628296ee99e service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:29.775861 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed88ca-dbf1-43ce-89ad-fde563f529a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:29.778410 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed88ca-dbf1-43ce-89ad-fde563f529a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:29.778410 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed88ca-dbf1-43ce-89ad-fde563f529a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:29.778410 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed88ca-dbf1-43ce-89ad-fde563f529a8 admin admin] PATH_INFO: `/v3/users/5fb224e1b40342258c24a747e965042d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:29.780231 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed88ca-dbf1-43ce-89ad-fde563f529a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fb224e1b40342258c24a747e965042d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:29.784430 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed88ca-dbf1-43ce-89ad-fde563f529a8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:30.026152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3ed88ca-dbf1-43ce-89ad-fde563f529a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fb224e1b40342258c24a747e965042d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:30.141418 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e0279-55fe-43f9-8042-8894f119db94 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:30.146088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-454e0279-55fe-43f9-8042-8894f119db94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:30.163237 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3ed88ca-dbf1-43ce-89ad-fde563f529a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fb224e1b40342258c24a747e965042d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:30.179707 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e0279-55fe-43f9-8042-8894f119db94 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:30.180331 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3ed88ca-dbf1-43ce-89ad-fde563f529a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fb224e1b40342258c24a747e965042d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:30.192350 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a9758-af36-4b1a-881e-4de812369d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:30.195657 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a9758-af36-4b1a-881e-4de812369d5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:30.196768 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a9758-af36-4b1a-881e-4de812369d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:30.197866 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a9758-af36-4b1a-881e-4de812369d5e admin admin] PATH_INFO: `/v3/users/4df400ec0c844bfdbad7589f405fd225` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:30.200458 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a9758-af36-4b1a-881e-4de812369d5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4df400ec0c844bfdbad7589f405fd225)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:30.207032 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a9758-af36-4b1a-881e-4de812369d5e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:30.213417 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3ed88ca-dbf1-43ce-89ad-fde563f529a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fb224e1b40342258c24a747e965042d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:30.214244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3ed88ca-dbf1-43ce-89ad-fde563f529a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fb224e1b40342258c24a747e965042d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:30.223566 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 943/1880] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:15:29 2022] DELETE /identity/v3/users/5fb224e1b40342258c24a747e965042d => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:30.533052 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-458a9758-af36-4b1a-881e-4de812369d5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4df400ec0c844bfdbad7589f405fd225'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:30.611351 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed88ca-dbf1-43ce-89ad-fde563f529a8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:30.613285 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-458a9758-af36-4b1a-881e-4de812369d5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4df400ec0c844bfdbad7589f405fd225'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:30.614759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a3ed88ca-dbf1-43ce-89ad-fde563f529a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:30.655389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-458a9758-af36-4b1a-881e-4de812369d5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4df400ec0c844bfdbad7589f405fd225'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:30.657125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-458a9758-af36-4b1a-881e-4de812369d5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4df400ec0c844bfdbad7589f405fd225'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:30.658744 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-458a9758-af36-4b1a-881e-4de812369d5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4df400ec0c844bfdbad7589f405fd225'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:30.694285 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 938/1881] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:15:30 2022] DELETE /identity/v3/users/4df400ec0c844bfdbad7589f405fd225 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:30.703071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7435bd-4fc4-4e05-92ef-e9b2aa7ac67d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:30.703547 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7435bd-4fc4-4e05-92ef-e9b2aa7ac67d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:30.704404 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7435bd-4fc4-4e05-92ef-e9b2aa7ac67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:30.744189 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:30.779348 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ed7435bd-4fc4-4e05-92ef-e9b2aa7ac67d None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:30.780373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ed7435bd-4fc4-4e05-92ef-e9b2aa7ac67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:30.861733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:30.903907 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed88ca-dbf1-43ce-89ad-fde563f529a8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:30.926014 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 939/1882] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:30.949375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7435bd-4fc4-4e05-92ef-e9b2aa7ac67d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:30.954030 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ed7435bd-4fc4-4e05-92ef-e9b2aa7ac67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:31.072076 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4898f-3faa-4f3c-a2ae-d4b319e5e988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:31.073153 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4898f-3faa-4f3c-a2ae-d4b319e5e988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:31.073645 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4898f-3faa-4f3c-a2ae-d4b319e5e988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:31.074140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4898f-3faa-4f3c-a2ae-d4b319e5e988 admin admin] PATH_INFO: `/v3/projects/a0b9c7f452594b2c92af6692ac64fde9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:31.076705 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4898f-3faa-4f3c-a2ae-d4b319e5e988 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0b9c7f452594b2c92af6692ac64fde9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:31.081116 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4898f-3faa-4f3c-a2ae-d4b319e5e988 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:31.116658 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7435bd-4fc4-4e05-92ef-e9b2aa7ac67d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:31.131258 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3975fc-9c20-443c-8a46-f04e4b41ee7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:31.133457 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3975fc-9c20-443c-8a46-f04e4b41ee7b service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:31.135337 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3975fc-9c20-443c-8a46-f04e4b41ee7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:31.137459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3975fc-9c20-443c-8a46-f04e4b41ee7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:31.172954 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3975fc-9c20-443c-8a46-f04e4b41ee7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:31.177607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3975fc-9c20-443c-8a46-f04e4b41ee7b service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:31.205156 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 940/1883] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:15:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:15:31.270608 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-23c4898f-3faa-4f3c-a2ae-d4b319e5e988 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0b9c7f452594b2c92af6692ac64fde9', 'request_id': 'req-23c4898f-3faa-4f3c-a2ae-d4b319e5e988'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:31.275133 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-23c4898f-3faa-4f3c-a2ae-d4b319e5e988 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0b9c7f452594b2c92af6692ac64fde9', 'request_id': 'req-23c4898f-3faa-4f3c-a2ae-d4b319e5e988'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:31.362531 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-23c4898f-3faa-4f3c-a2ae-d4b319e5e988 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0b9c7f452594b2c92af6692ac64fde9', 'request_id': 'req-23c4898f-3faa-4f3c-a2ae-d4b319e5e988'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:31.385533 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-23c4898f-3faa-4f3c-a2ae-d4b319e5e988 admin admin] The token cache is being invalidate because project a0b9c7f452594b2c92af6692ac64fde9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:15:31.386230 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-23c4898f-3faa-4f3c-a2ae-d4b319e5e988 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:31.391138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-23c4898f-3faa-4f3c-a2ae-d4b319e5e988 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:31.396024 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 944/1884] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:15:30 2022] DELETE /identity/v3/projects/a0b9c7f452594b2c92af6692ac64fde9 => generated 0 bytes in 786 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:31.403234 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3975fc-9c20-443c-8a46-f04e4b41ee7b service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:31.406455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3b3975fc-9c20-443c-8a46-f04e4b41ee7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:31.440370 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3975fc-9c20-443c-8a46-f04e4b41ee7b service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:31.533825 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4898f-3faa-4f3c-a2ae-d4b319e5e988 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:31.543684 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-23c4898f-3faa-4f3c-a2ae-d4b319e5e988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:31.554865 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459435d-087a-4092-9530-051ebfe45bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:31.556110 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459435d-087a-4092-9530-051ebfe45bea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:31.556696 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459435d-087a-4092-9530-051ebfe45bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:31.557266 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459435d-087a-4092-9530-051ebfe45bea admin admin] PATH_INFO: `/v3/users/261c6626328747a49186115df8dc8a97` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:31.559467 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459435d-087a-4092-9530-051ebfe45bea admin admin] RBAC: Authorizing `identity:delete_user(user_id=261c6626328747a49186115df8dc8a97)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:31.563785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459435d-087a-4092-9530-051ebfe45bea admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:31.570976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4898f-3faa-4f3c-a2ae-d4b319e5e988 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:31.579812 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd74e3-427e-4634-a668-d22503003015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:31.584915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd74e3-427e-4634-a668-d22503003015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:31.584915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd74e3-427e-4634-a668-d22503003015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:31.584915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd74e3-427e-4634-a668-d22503003015 admin admin] PATH_INFO: `/v3/projects/7fc28a3214084728bbfb0d0fbc5a2902` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:31.585774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd74e3-427e-4634-a668-d22503003015 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fc28a3214084728bbfb0d0fbc5a2902)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:31.589705 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd74e3-427e-4634-a668-d22503003015 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:31.857406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-61dd74e3-427e-4634-a668-d22503003015 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fc28a3214084728bbfb0d0fbc5a2902', 'request_id': 'req-61dd74e3-427e-4634-a668-d22503003015'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:31.859452 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9459435d-087a-4092-9530-051ebfe45bea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '261c6626328747a49186115df8dc8a97'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:31.874430 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-61dd74e3-427e-4634-a668-d22503003015 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fc28a3214084728bbfb0d0fbc5a2902', 'request_id': 'req-61dd74e3-427e-4634-a668-d22503003015'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:31.887682 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:32.007553 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9459435d-087a-4092-9530-051ebfe45bea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '261c6626328747a49186115df8dc8a97'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:32.034482 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9459435d-087a-4092-9530-051ebfe45bea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '261c6626328747a49186115df8dc8a97'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:32.035678 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9459435d-087a-4092-9530-051ebfe45bea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '261c6626328747a49186115df8dc8a97'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:32.036796 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9459435d-087a-4092-9530-051ebfe45bea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '261c6626328747a49186115df8dc8a97'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:32.047932 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:32.062324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 941/1885] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:15:31 2022] DELETE /identity/v3/users/261c6626328747a49186115df8dc8a97 => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:32.063769 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459435d-087a-4092-9530-051ebfe45bea admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:32.066031 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9459435d-087a-4092-9530-051ebfe45bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:32.122176 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-61dd74e3-427e-4634-a668-d22503003015 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fc28a3214084728bbfb0d0fbc5a2902', 'request_id': 'req-61dd74e3-427e-4634-a668-d22503003015'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:32.147168 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-61dd74e3-427e-4634-a668-d22503003015 admin admin] The token cache is being invalidate because project 7fc28a3214084728bbfb0d0fbc5a2902 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:15:32.147168 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-61dd74e3-427e-4634-a668-d22503003015 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:32.148373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-61dd74e3-427e-4634-a668-d22503003015 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:32.151739 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 945/1886] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:15:31 2022] DELETE /identity/v3/projects/7fc28a3214084728bbfb0d0fbc5a2902 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:32.161789 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451e1b0-3d36-45d3-ad3e-30dcd1d3de16 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:32.174168 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451e1b0-3d36-45d3-ad3e-30dcd1d3de16 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:32.174168 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451e1b0-3d36-45d3-ad3e-30dcd1d3de16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:32.209866 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a451e1b0-3d36-45d3-ad3e-30dcd1d3de16 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:32.212317 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a451e1b0-3d36-45d3-ad3e-30dcd1d3de16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:32.240982 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 946/1887] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:32.248364 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459435d-087a-4092-9530-051ebfe45bea admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:32.248364 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa806b53-2ef2-4ee0-97a7-66fa720c15c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:32.248364 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa806b53-2ef2-4ee0-97a7-66fa720c15c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:32.248364 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa806b53-2ef2-4ee0-97a7-66fa720c15c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:32.249539 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa806b53-2ef2-4ee0-97a7-66fa720c15c2 admin admin] PATH_INFO: `/v3/projects/7216835c713b4b45b0c060b32e44608d` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:32.249539 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa806b53-2ef2-4ee0-97a7-66fa720c15c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7216835c713b4b45b0c060b32e44608d)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:32.249792 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee816b-bac7-4d5a-a286-56c9fcb2f22f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:32.249792 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee816b-bac7-4d5a-a286-56c9fcb2f22f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:32.249792 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee816b-bac7-4d5a-a286-56c9fcb2f22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:32.250969 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa806b53-2ef2-4ee0-97a7-66fa720c15c2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:32.274056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-10ee816b-bac7-4d5a-a286-56c9fcb2f22f None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:32.275801 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-10ee816b-bac7-4d5a-a286-56c9fcb2f22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:32.285252 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 947/1888] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:15:32.289709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee816b-bac7-4d5a-a286-56c9fcb2f22f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:32.299874 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee816b-bac7-4d5a-a286-56c9fcb2f22f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:32.307060 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7164cf5-82d1-483e-83f7-50094e16f85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:32.307333 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7164cf5-82d1-483e-83f7-50094e16f85f admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:32.310717 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7164cf5-82d1-483e-83f7-50094e16f85f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:32.310717 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7164cf5-82d1-483e-83f7-50094e16f85f admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:32.310717 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7164cf5-82d1-483e-83f7-50094e16f85f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:15:32.310717 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7164cf5-82d1-483e-83f7-50094e16f85f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:32.311336 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7164cf5-82d1-483e-83f7-50094e16f85f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:32.340021 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 948/1889] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 17:15:32.346318 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7164cf5-82d1-483e-83f7-50094e16f85f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:32.367933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7164cf5-82d1-483e-83f7-50094e16f85f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:32.382225 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f67af1-955e-434d-99fa-f5de3ee09c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:32.383823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f67af1-955e-434d-99fa-f5de3ee09c0b admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:32.386126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f67af1-955e-434d-99fa-f5de3ee09c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:32.394980 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f67af1-955e-434d-99fa-f5de3ee09c0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:32.401689 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f67af1-955e-434d-99fa-f5de3ee09c0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:32.416405 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f67af1-955e-434d-99fa-f5de3ee09c0b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:32.449987 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-aa806b53-2ef2-4ee0-97a7-66fa720c15c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7216835c713b4b45b0c060b32e44608d', 'request_id': 'req-aa806b53-2ef2-4ee0-97a7-66fa720c15c2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:32.550163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-aa806b53-2ef2-4ee0-97a7-66fa720c15c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7216835c713b4b45b0c060b32e44608d', 'request_id': 'req-aa806b53-2ef2-4ee0-97a7-66fa720c15c2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:32.566326 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-aa806b53-2ef2-4ee0-97a7-66fa720c15c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7216835c713b4b45b0c060b32e44608d', 'request_id': 'req-aa806b53-2ef2-4ee0-97a7-66fa720c15c2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:32.579285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 949/1890] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:15:32 2022] POST /identity/v3/projects => generated 383 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:15:32.585775 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f67af1-955e-434d-99fa-f5de3ee09c0b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:32.598115 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-aa806b53-2ef2-4ee0-97a7-66fa720c15c2 admin admin] The token cache is being invalidate because project 7216835c713b4b45b0c060b32e44608d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:15:32.598589 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-aa806b53-2ef2-4ee0-97a7-66fa720c15c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:32.599448 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-aa806b53-2ef2-4ee0-97a7-66fa720c15c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:32.601062 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 942/1891] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:15:32 2022] DELETE /identity/v3/projects/7216835c713b4b45b0c060b32e44608d => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:32.607317 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa806b53-2ef2-4ee0-97a7-66fa720c15c2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:32.619435 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-aa806b53-2ef2-4ee0-97a7-66fa720c15c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:32.762539 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa806b53-2ef2-4ee0-97a7-66fa720c15c2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:32.766762 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f67af1-955e-434d-99fa-f5de3ee09c0b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:32.768424 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0daf8ec-cfd5-4f1b-abd8-8f4d4b33b67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:32.769080 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0daf8ec-cfd5-4f1b-abd8-8f4d4b33b67b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:32.771467 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0daf8ec-cfd5-4f1b-abd8-8f4d4b33b67b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:32.771467 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0daf8ec-cfd5-4f1b-abd8-8f4d4b33b67b admin admin] PATH_INFO: `/v3/users/e1721f8e3adf4c499af2f27577114cf3` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:32.771467 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0daf8ec-cfd5-4f1b-abd8-8f4d4b33b67b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1721f8e3adf4c499af2f27577114cf3)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:32.773297 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0daf8ec-cfd5-4f1b-abd8-8f4d4b33b67b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:32.781867 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b8e63-9b36-4b93-b449-d140ee9591bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:32.784807 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b8e63-9b36-4b93-b449-d140ee9591bd admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:32.786374 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b8e63-9b36-4b93-b449-d140ee9591bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:32.787040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b8e63-9b36-4b93-b449-d140ee9591bd admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:32.791738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b8e63-9b36-4b93-b449-d140ee9591bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:32.799263 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b8e63-9b36-4b93-b449-d140ee9591bd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:32.923284 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:32.965215 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 950/1892] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:15:32 2022] POST /identity/v3/users => generated 292 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:15:32.969038 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b8e63-9b36-4b93-b449-d140ee9591bd admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:32.989024 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7f6b8e63-9b36-4b93-b449-d140ee9591bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:33.023178 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b8e63-9b36-4b93-b449-d140ee9591bd admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:33.036447 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d841b8-5a58-42e0-8521-2bab5e7d589d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:33.045199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d841b8-5a58-42e0-8521-2bab5e7d589d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:33.045199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d841b8-5a58-42e0-8521-2bab5e7d589d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:33.045199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d841b8-5a58-42e0-8521-2bab5e7d589d admin admin] PATH_INFO: `/v3/projects/fe6506d2bf8c4e508343e9e2983c5e28` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:33.045199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d841b8-5a58-42e0-8521-2bab5e7d589d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe6506d2bf8c4e508343e9e2983c5e28)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:33.047294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d841b8-5a58-42e0-8521-2bab5e7d589d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:33.055490 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d0daf8ec-cfd5-4f1b-abd8-8f4d4b33b67b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1721f8e3adf4c499af2f27577114cf3'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:33.200274 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d0daf8ec-cfd5-4f1b-abd8-8f4d4b33b67b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1721f8e3adf4c499af2f27577114cf3'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:33.223574 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d0daf8ec-cfd5-4f1b-abd8-8f4d4b33b67b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1721f8e3adf4c499af2f27577114cf3'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:33.224532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d0daf8ec-cfd5-4f1b-abd8-8f4d4b33b67b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1721f8e3adf4c499af2f27577114cf3'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:33.225248 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d0daf8ec-cfd5-4f1b-abd8-8f4d4b33b67b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1721f8e3adf4c499af2f27577114cf3'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:33.246316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 943/1893] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:15:32 2022] DELETE /identity/v3/users/e1721f8e3adf4c499af2f27577114cf3 => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:15:33.251267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0daf8ec-cfd5-4f1b-abd8-8f4d4b33b67b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:33.251267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d0daf8ec-cfd5-4f1b-abd8-8f4d4b33b67b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:33.323926 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3d841b8-5a58-42e0-8521-2bab5e7d589d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe6506d2bf8c4e508343e9e2983c5e28', 'request_id': 'req-a3d841b8-5a58-42e0-8521-2bab5e7d589d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:33.345677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3d841b8-5a58-42e0-8521-2bab5e7d589d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe6506d2bf8c4e508343e9e2983c5e28', 'request_id': 'req-a3d841b8-5a58-42e0-8521-2bab5e7d589d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:33.383896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0daf8ec-cfd5-4f1b-abd8-8f4d4b33b67b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:33.424797 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3d841b8-5a58-42e0-8521-2bab5e7d589d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe6506d2bf8c4e508343e9e2983c5e28', 'request_id': 'req-a3d841b8-5a58-42e0-8521-2bab5e7d589d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:33.461162 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3d841b8-5a58-42e0-8521-2bab5e7d589d admin admin] The token cache is being invalidate because project fe6506d2bf8c4e508343e9e2983c5e28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:15:33.461634 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3d841b8-5a58-42e0-8521-2bab5e7d589d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:33.465796 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3d841b8-5a58-42e0-8521-2bab5e7d589d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:33.472199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 951/1894] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:15:32 2022] DELETE /identity/v3/projects/fe6506d2bf8c4e508343e9e2983c5e28 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:33.477743 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839082c8-d00a-43e2-9a82-ddfc697b7aaa None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:33.478237 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839082c8-d00a-43e2-9a82-ddfc697b7aaa None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:33.478730 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839082c8-d00a-43e2-9a82-ddfc697b7aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:33.520049 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-839082c8-d00a-43e2-9a82-ddfc697b7aaa None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:33.522636 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-839082c8-d00a-43e2-9a82-ddfc697b7aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:33.532316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=10,keystone:SELECT=85,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:33.538981 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ba699-3a3b-4d40-bdf4-a22fd27c4503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:33.540055 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ba699-3a3b-4d40-bdf4-a22fd27c4503 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:33.540580 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ba699-3a3b-4d40-bdf4-a22fd27c4503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:33.541029 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ba699-3a3b-4d40-bdf4-a22fd27c4503 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:33.542187 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ba699-3a3b-4d40-bdf4-a22fd27c4503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:33.545335 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ba699-3a3b-4d40-bdf4-a22fd27c4503 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:33.568959 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 944/1895] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:15:33.575918 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ba699-3a3b-4d40-bdf4-a22fd27c4503 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:33.578924 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a25ba699-3a3b-4d40-bdf4-a22fd27c4503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:33.603501 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ba699-3a3b-4d40-bdf4-a22fd27c4503 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:33.631961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa445c6-e999-4feb-8980-1d659aa31d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:33.632919 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa445c6-e999-4feb-8980-1d659aa31d8d admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:33.633422 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa445c6-e999-4feb-8980-1d659aa31d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:33.633765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa445c6-e999-4feb-8980-1d659aa31d8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:33.635752 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa445c6-e999-4feb-8980-1d659aa31d8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:33.638715 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa445c6-e999-4feb-8980-1d659aa31d8d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:33.655104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 945/1896] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:15:33.661064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa445c6-e999-4feb-8980-1d659aa31d8d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:33.687905 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa445c6-e999-4feb-8980-1d659aa31d8d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:33.701235 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131160a-1410-48c3-8182-02ce185cf89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:33.701923 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131160a-1410-48c3-8182-02ce185cf89c admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:33.702201 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131160a-1410-48c3-8182-02ce185cf89c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:33.702501 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131160a-1410-48c3-8182-02ce185cf89c admin admin] PATH_INFO: `/v3/projects/26a9d512d3be400d89d90bce5d3e62fc/users/5bef640e14af4d1cadce2e17296be286/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:33.727428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 952/1897] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:33.738155 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7131160a-1410-48c3-8182-02ce185cf89c admin admin] RBAC: Authorizing `identity:create_grant(project_id=26a9d512d3be400d89d90bce5d3e62fc, user_id=5bef640e14af4d1cadce2e17296be286, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:33.739678 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5b9c2-c8ba-4db9-989d-86550f7b8ae4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:33.740301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5b9c2-c8ba-4db9-989d-86550f7b8ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:33.740912 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5b9c2-c8ba-4db9-989d-86550f7b8ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:33.745802 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7131160a-1410-48c3-8182-02ce185cf89c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:33.796271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-edb5b9c2-c8ba-4db9-989d-86550f7b8ae4 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:33.798755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-edb5b9c2-c8ba-4db9-989d-86550f7b8ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:33.816576 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 953/1898] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:33.818882 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5b9c2-c8ba-4db9-989d-86550f7b8ae4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:33.840146 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 946/1899] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:15:33 2022] PUT /identity/v3/projects/26a9d512d3be400d89d90bce5d3e62fc/users/5bef640e14af4d1cadce2e17296be286/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:15:33.848217 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5b9c2-c8ba-4db9-989d-86550f7b8ae4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:33.853391 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf07730-f36c-4fa7-876f-992d1e6038af None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:33.855243 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf07730-f36c-4fa7-876f-992d1e6038af None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:33.855545 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf07730-f36c-4fa7-876f-992d1e6038af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:33.905372 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:33.917453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-baf07730-f36c-4fa7-876f-992d1e6038af None None] MFA Rules not processed for user `5bef640e14af4d1cadce2e17296be286`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:33.926825 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-baf07730-f36c-4fa7-876f-992d1e6038af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:33.973983 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc8b98-9af7-47e8-abc9-09eacf336c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:33.974742 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc8b98-9af7-47e8-abc9-09eacf336c28 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:33.975025 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc8b98-9af7-47e8-abc9-09eacf336c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:33.975301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc8b98-9af7-47e8-abc9-09eacf336c28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:33.975974 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecc8b98-9af7-47e8-abc9-09eacf336c28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:15:33.976279 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecc8b98-9af7-47e8-abc9-09eacf336c28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:33.978915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecc8b98-9af7-47e8-abc9-09eacf336c28 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:33.994830 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 954/1900] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:15:34.001814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc8b98-9af7-47e8-abc9-09eacf336c28 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:34.017464 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc8b98-9af7-47e8-abc9-09eacf336c28 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:34.024094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd157e-c3b9-42bd-8157-82cb29b0b709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:34.025257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd157e-c3b9-42bd-8157-82cb29b0b709 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:34.025604 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd157e-c3b9-42bd-8157-82cb29b0b709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:34.025931 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd157e-c3b9-42bd-8157-82cb29b0b709 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:34.026612 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cd157e-c3b9-42bd-8157-82cb29b0b709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:34.043978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cd157e-c3b9-42bd-8157-82cb29b0b709 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:34.113618 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 947/1901] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:34.119304 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf07730-f36c-4fa7-876f-992d1e6038af None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:34.150485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf07730-f36c-4fa7-876f-992d1e6038af None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:34.158849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2fbd19-a7de-49fb-9966-ef660c2bc7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:34.160641 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fbd19-a7de-49fb-9966-ef660c2bc7de admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:34.160641 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fbd19-a7de-49fb-9966-ef660c2bc7de admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:34.161048 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fbd19-a7de-49fb-9966-ef660c2bc7de admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:34.162288 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2fbd19-a7de-49fb-9966-ef660c2bc7de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:34.167049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2fbd19-a7de-49fb-9966-ef660c2bc7de admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:34.190074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 955/1902] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:15:34 2022] POST /identity/v3/projects => generated 365 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:15:34.194198 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd157e-c3b9-42bd-8157-82cb29b0b709 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:34.291952 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 948/1903] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:15:34 2022] POST /identity/v3/projects => generated 383 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:15:34.297699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2fbd19-a7de-49fb-9966-ef660c2bc7de admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:34.403918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd157e-c3b9-42bd-8157-82cb29b0b709 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:34.406623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2fbd19-a7de-49fb-9966-ef660c2bc7de admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:34.415568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b76ad-6446-48b4-944d-e19cb4d050f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:34.417002 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b76ad-6446-48b4-944d-e19cb4d050f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:34.417002 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b76ad-6446-48b4-944d-e19cb4d050f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:34.417451 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b76ad-6446-48b4-944d-e19cb4d050f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:34.418210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b76ad-6446-48b4-944d-e19cb4d050f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:34.425381 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b76ad-6446-48b4-944d-e19cb4d050f9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:34.425604 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3e61d-cb2a-41b4-897a-ea00a5a40c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:34.425604 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3e61d-cb2a-41b4-897a-ea00a5a40c9b admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:34.425986 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3e61d-cb2a-41b4-897a-ea00a5a40c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:34.428715 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3e61d-cb2a-41b4-897a-ea00a5a40c9b admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:34.429581 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3e61d-cb2a-41b4-897a-ea00a5a40c9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:34.435367 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3e61d-cb2a-41b4-897a-ea00a5a40c9b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:34.546951 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 956/1904] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:15:34 2022] POST /identity/v3/users => generated 283 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:15:34.548682 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 949/1905] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:15:34 2022] POST /identity/v3/users => generated 298 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:15:34.549623 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3e61d-cb2a-41b4-897a-ea00a5a40c9b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:34.550887 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b76ad-6446-48b4-944d-e19cb4d050f9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:34.559691 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:34.564883 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3e61d-cb2a-41b4-897a-ea00a5a40c9b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:34.565699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b76ad-6446-48b4-944d-e19cb4d050f9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:34.575678 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b251e7-47ae-4b15-baa5-ab69d18c0e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:34.576508 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc915bad-8d98-439f-af9f-88b274ea719b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:34.576715 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b251e7-47ae-4b15-baa5-ab69d18c0e58 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:34.576993 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b251e7-47ae-4b15-baa5-ab69d18c0e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:34.577395 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b251e7-47ae-4b15-baa5-ab69d18c0e58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:34.577569 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc915bad-8d98-439f-af9f-88b274ea719b admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:34.577808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc915bad-8d98-439f-af9f-88b274ea719b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:34.578155 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc915bad-8d98-439f-af9f-88b274ea719b admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:34.578594 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b251e7-47ae-4b15-baa5-ab69d18c0e58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:34.579339 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc915bad-8d98-439f-af9f-88b274ea719b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:34.580650 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b251e7-47ae-4b15-baa5-ab69d18c0e58 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:34.581439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc915bad-8d98-439f-af9f-88b274ea719b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:34.594937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 957/1906] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:15:34.598669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 950/1907] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:15:34.611569 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b251e7-47ae-4b15-baa5-ab69d18c0e58 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:34.619977 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc915bad-8d98-439f-af9f-88b274ea719b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:34.623660 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-80b251e7-47ae-4b15-baa5-ab69d18c0e58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:34.640863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b251e7-47ae-4b15-baa5-ab69d18c0e58 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:34.649642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc915bad-8d98-439f-af9f-88b274ea719b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:34.651848 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d24d2c-7eba-49fc-a8d0-b2f1a50142b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:34.652628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d24d2c-7eba-49fc-a8d0-b2f1a50142b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:34.652950 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d24d2c-7eba-49fc-a8d0-b2f1a50142b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:34.653180 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d24d2c-7eba-49fc-a8d0-b2f1a50142b6 admin admin] PATH_INFO: `/v3/projects/8e16765cf8b44c66823a4a4cf9862b99` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:34.655044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d24d2c-7eba-49fc-a8d0-b2f1a50142b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e16765cf8b44c66823a4a4cf9862b99)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:34.658514 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d24d2c-7eba-49fc-a8d0-b2f1a50142b6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:34.667341 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3bb971-9695-416b-bd97-b327d28a9f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:34.667965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3bb971-9695-416b-bd97-b327d28a9f50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:34.668215 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3bb971-9695-416b-bd97-b327d28a9f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:34.668480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3bb971-9695-416b-bd97-b327d28a9f50 admin admin] PATH_INFO: `/v3/projects/1593aaf958404b058550332d32e2d0af/users/ec32421a4fd34906be26af4a03fa49d4/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:34.700700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3bb971-9695-416b-bd97-b327d28a9f50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1593aaf958404b058550332d32e2d0af, user_id=ec32421a4fd34906be26af4a03fa49d4, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:34.705917 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3bb971-9695-416b-bd97-b327d28a9f50 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:34.797027 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 951/1908] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:15:34 2022] PUT /identity/v3/projects/1593aaf958404b058550332d32e2d0af/users/ec32421a4fd34906be26af4a03fa49d4/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:34.798036 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3bb971-9695-416b-bd97-b327d28a9f50 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:34.817111 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e4d24d2c-7eba-49fc-a8d0-b2f1a50142b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e16765cf8b44c66823a4a4cf9862b99', 'request_id': 'req-e4d24d2c-7eba-49fc-a8d0-b2f1a50142b6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:34.825910 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e4d24d2c-7eba-49fc-a8d0-b2f1a50142b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e16765cf8b44c66823a4a4cf9862b99', 'request_id': 'req-e4d24d2c-7eba-49fc-a8d0-b2f1a50142b6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:34.859604 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:34.901735 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e4d24d2c-7eba-49fc-a8d0-b2f1a50142b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e16765cf8b44c66823a4a4cf9862b99', 'request_id': 'req-e4d24d2c-7eba-49fc-a8d0-b2f1a50142b6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:34.909873 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3bb971-9695-416b-bd97-b327d28a9f50 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:34.925642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e4d24d2c-7eba-49fc-a8d0-b2f1a50142b6 admin admin] The token cache is being invalidate because project 8e16765cf8b44c66823a4a4cf9862b99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:15:34.925995 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e4d24d2c-7eba-49fc-a8d0-b2f1a50142b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:34.926731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e4d24d2c-7eba-49fc-a8d0-b2f1a50142b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:34.928922 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 958/1909] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:15:34 2022] DELETE /identity/v3/projects/8e16765cf8b44c66823a4a4cf9862b99 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:34.929877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d24d2c-7eba-49fc-a8d0-b2f1a50142b6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:34.933216 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e4d24d2c-7eba-49fc-a8d0-b2f1a50142b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:35.021462 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5e27d-da37-44ed-a147-fa196e54fbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:35.023137 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5e27d-da37-44ed-a147-fa196e54fbd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:35.023802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d24d2c-7eba-49fc-a8d0-b2f1a50142b6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:35.025791 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5e27d-da37-44ed-a147-fa196e54fbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:35.026611 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5e27d-da37-44ed-a147-fa196e54fbd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:35.027998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5e27d-da37-44ed-a147-fa196e54fbd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:35.031296 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5e27d-da37-44ed-a147-fa196e54fbd8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:35.034529 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bfb98-4c55-4f2c-89f3-d644a01f7424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:35.035662 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bfb98-4c55-4f2c-89f3-d644a01f7424 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:35.036167 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bfb98-4c55-4f2c-89f3-d644a01f7424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:35.036607 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bfb98-4c55-4f2c-89f3-d644a01f7424 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:35.038699 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26bfb98-4c55-4f2c-89f3-d644a01f7424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:35.042641 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26bfb98-4c55-4f2c-89f3-d644a01f7424 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:35.052028 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 952/1910] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:15:35.053313 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5e27d-da37-44ed-a147-fa196e54fbd8 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:35.057946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-72d5e27d-da37-44ed-a147-fa196e54fbd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:35.060597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 959/1911] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:15:35.061453 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bfb98-4c55-4f2c-89f3-d644a01f7424 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:35.063233 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b26bfb98-4c55-4f2c-89f3-d644a01f7424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:35.074866 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5e27d-da37-44ed-a147-fa196e54fbd8 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:35.077932 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bfb98-4c55-4f2c-89f3-d644a01f7424 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:35.080759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768550d-ff70-4d90-a34e-e729bd20428d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:35.081694 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768550d-ff70-4d90-a34e-e729bd20428d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:35.082246 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768550d-ff70-4d90-a34e-e729bd20428d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:35.082681 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768550d-ff70-4d90-a34e-e729bd20428d admin admin] PATH_INFO: `/v3/users/be4b056a23a74e558709c86ad9e7abd6` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:35.083334 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e3a21-af2c-4ed3-98f1-a6dd25aa416c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:35.084001 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e3a21-af2c-4ed3-98f1-a6dd25aa416c admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:35.084176 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768550d-ff70-4d90-a34e-e729bd20428d admin admin] RBAC: Authorizing `identity:delete_user(user_id=be4b056a23a74e558709c86ad9e7abd6)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:35.084337 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e3a21-af2c-4ed3-98f1-a6dd25aa416c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:35.084529 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e3a21-af2c-4ed3-98f1-a6dd25aa416c admin admin] PATH_INFO: `/v3/projects/4f89a6525d6742fd8b62dd380dc17418/users/5b00585340bf4ff380911677605160bd/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:35.086790 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768550d-ff70-4d90-a34e-e729bd20428d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:35.108290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e3a21-af2c-4ed3-98f1-a6dd25aa416c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f89a6525d6742fd8b62dd380dc17418, user_id=5b00585340bf4ff380911677605160bd, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:35.111531 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e3a21-af2c-4ed3-98f1-a6dd25aa416c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:35.218498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 960/1912] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:15:35 2022] PUT /identity/v3/projects/4f89a6525d6742fd8b62dd380dc17418/users/5b00585340bf4ff380911677605160bd/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:35.237885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e3a21-af2c-4ed3-98f1-a6dd25aa416c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:35.357475 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:35.386086 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e3a21-af2c-4ed3-98f1-a6dd25aa416c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:35.395972 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c161b08-180c-4593-8f4c-2ff18a6520f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:35.397063 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c161b08-180c-4593-8f4c-2ff18a6520f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:35.401945 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c161b08-180c-4593-8f4c-2ff18a6520f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:35.401945 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c161b08-180c-4593-8f4c-2ff18a6520f5 admin admin] PATH_INFO: `/v3/projects/1593aaf958404b058550332d32e2d0af/users/ec32421a4fd34906be26af4a03fa49d4/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:35.401945 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c161b08-180c-4593-8f4c-2ff18a6520f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1593aaf958404b058550332d32e2d0af, user_id=ec32421a4fd34906be26af4a03fa49d4, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:35.403461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f768550d-ff70-4d90-a34e-e729bd20428d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be4b056a23a74e558709c86ad9e7abd6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:35.406403 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c161b08-180c-4593-8f4c-2ff18a6520f5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:35.432789 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 961/1913] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:15:35 2022] PUT /identity/v3/projects/1593aaf958404b058550332d32e2d0af/users/ec32421a4fd34906be26af4a03fa49d4/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:35.436608 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9209e0-534e-41f7-88da-c12060723d93 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:35.437093 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9209e0-534e-41f7-88da-c12060723d93 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:35.438813 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9209e0-534e-41f7-88da-c12060723d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:35.476480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f768550d-ff70-4d90-a34e-e729bd20428d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be4b056a23a74e558709c86ad9e7abd6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:35.503828 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7f9209e0-534e-41f7-88da-c12060723d93 None None] MFA Rules not processed for user `5b00585340bf4ff380911677605160bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:35.505275 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7f9209e0-534e-41f7-88da-c12060723d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:35.507576 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f768550d-ff70-4d90-a34e-e729bd20428d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be4b056a23a74e558709c86ad9e7abd6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:35.509957 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f768550d-ff70-4d90-a34e-e729bd20428d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be4b056a23a74e558709c86ad9e7abd6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:35.515447 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f768550d-ff70-4d90-a34e-e729bd20428d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be4b056a23a74e558709c86ad9e7abd6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:35.559832 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 953/1914] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:15:35 2022] DELETE /identity/v3/users/be4b056a23a74e558709c86ad9e7abd6 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:35.565572 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd0c6e-d10d-4636-a6bd-cdf35be1d3e6 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:35.565572 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd0c6e-d10d-4636-a6bd-cdf35be1d3e6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:35.565572 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd0c6e-d10d-4636-a6bd-cdf35be1d3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:35.620362 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-bedd0c6e-d10d-4636-a6bd-cdf35be1d3e6 None None] MFA Rules not processed for user `ec32421a4fd34906be26af4a03fa49d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:35.621441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bedd0c6e-d10d-4636-a6bd-cdf35be1d3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:35.669590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 962/1915] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:35.678343 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9209e0-534e-41f7-88da-c12060723d93 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:35.680907 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7f9209e0-534e-41f7-88da-c12060723d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:35.732114 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:35.829592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 954/1916] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:35 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:35.849100 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd0c6e-d10d-4636-a6bd-cdf35be1d3e6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:35.850814 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bedd0c6e-d10d-4636-a6bd-cdf35be1d3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:35.857339 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9209e0-534e-41f7-88da-c12060723d93 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:35.943807 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd0c6e-d10d-4636-a6bd-cdf35be1d3e6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:35.948384 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bbe17-5e9a-4e89-9718-2ecc1ada107e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:35.949808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bbe17-5e9a-4e89-9718-2ecc1ada107e service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:35.949808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bbe17-5e9a-4e89-9718-2ecc1ada107e service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:35.950147 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bbe17-5e9a-4e89-9718-2ecc1ada107e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:35.952731 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-322bbe17-5e9a-4e89-9718-2ecc1ada107e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:35.975579 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0fe10-dc6b-48d0-bcf8-3b3ad93937a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:35.977287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0fe10-dc6b-48d0-bcf8-3b3ad93937a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:35.977616 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0fe10-dc6b-48d0-bcf8-3b3ad93937a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:35.978486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0fe10-dc6b-48d0-bcf8-3b3ad93937a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:35.979156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c0fe10-dc6b-48d0-bcf8-3b3ad93937a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:35.981913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c0fe10-dc6b-48d0-bcf8-3b3ad93937a0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:36.081361 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322bbe17-5e9a-4e89-9718-2ecc1ada107e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:36.082724 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322bbe17-5e9a-4e89-9718-2ecc1ada107e service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:36.096672 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 955/1917] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:15:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:15:36.159578 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 963/1918] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:15:35 2022] POST /identity/v3/projects => generated 365 bytes in 482 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:15:36.166721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bbe17-5e9a-4e89-9718-2ecc1ada107e service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:36.190521 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bbe17-5e9a-4e89-9718-2ecc1ada107e service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:36.367785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6745561-8843-4ebb-8ef3-77c204dbd329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:36.369597 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6745561-8843-4ebb-8ef3-77c204dbd329 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:36.370209 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6745561-8843-4ebb-8ef3-77c204dbd329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:36.371036 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6745561-8843-4ebb-8ef3-77c204dbd329 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:36.372342 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6745561-8843-4ebb-8ef3-77c204dbd329 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:36.377069 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6745561-8843-4ebb-8ef3-77c204dbd329 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:36.455347 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0fe10-dc6b-48d0-bcf8-3b3ad93937a0 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:36.461839 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-77c0fe10-dc6b-48d0-bcf8-3b3ad93937a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:36.474700 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:36.496270 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0fe10-dc6b-48d0-bcf8-3b3ad93937a0 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:36.503467 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 956/1919] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:15:36 2022] POST /identity/v3/users => generated 289 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:15:36.505276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6745561-8843-4ebb-8ef3-77c204dbd329 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:36.520831 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97f64b-e610-4acc-8aef-85305cd1c83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:36.521859 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97f64b-e610-4acc-8aef-85305cd1c83f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:36.522324 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97f64b-e610-4acc-8aef-85305cd1c83f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:36.527050 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97f64b-e610-4acc-8aef-85305cd1c83f admin admin] PATH_INFO: `/v3/projects/a575455a3d2f4277bae898425ebbee06` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:36.530592 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97f64b-e610-4acc-8aef-85305cd1c83f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a575455a3d2f4277bae898425ebbee06)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:36.536953 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97f64b-e610-4acc-8aef-85305cd1c83f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:36.543756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6745561-8843-4ebb-8ef3-77c204dbd329 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:36.562064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267ed5d-b426-4434-ba01-8492253f9396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:36.563103 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267ed5d-b426-4434-ba01-8492253f9396 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:36.563574 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267ed5d-b426-4434-ba01-8492253f9396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:36.564013 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267ed5d-b426-4434-ba01-8492253f9396 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:36.565436 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b267ed5d-b426-4434-ba01-8492253f9396 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:36.568538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b267ed5d-b426-4434-ba01-8492253f9396 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:36.588761 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 957/1920] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:15:36.594821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267ed5d-b426-4434-ba01-8492253f9396 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:36.620549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267ed5d-b426-4434-ba01-8492253f9396 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:36.629131 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41841f0c-f54f-4e86-a6ec-e49acb0f4f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:36.630103 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41841f0c-f54f-4e86-a6ec-e49acb0f4f5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:36.630821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41841f0c-f54f-4e86-a6ec-e49acb0f4f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:36.631306 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41841f0c-f54f-4e86-a6ec-e49acb0f4f5e admin admin] PATH_INFO: `/v3/projects/9df03841d74a4020a5b917ef843c54f8/users/ffe222654333427e92f261b56f4e9c8f/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:36.636385 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:36.660456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41841f0c-f54f-4e86-a6ec-e49acb0f4f5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9df03841d74a4020a5b917ef843c54f8, user_id=ffe222654333427e92f261b56f4e9c8f, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:36.665930 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41841f0c-f54f-4e86-a6ec-e49acb0f4f5e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:36.746146 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 958/1921] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:15:36 2022] PUT /identity/v3/projects/9df03841d74a4020a5b917ef843c54f8/users/ffe222654333427e92f261b56f4e9c8f/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:36.750126 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41841f0c-f54f-4e86-a6ec-e49acb0f4f5e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:36.762575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6f97f64b-e610-4acc-8aef-85305cd1c83f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a575455a3d2f4277bae898425ebbee06', 'request_id': 'req-6f97f64b-e610-4acc-8aef-85305cd1c83f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:36.767759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6f97f64b-e610-4acc-8aef-85305cd1c83f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a575455a3d2f4277bae898425ebbee06', 'request_id': 'req-6f97f64b-e610-4acc-8aef-85305cd1c83f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:36.848832 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41841f0c-f54f-4e86-a6ec-e49acb0f4f5e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:36.857497 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6f97f64b-e610-4acc-8aef-85305cd1c83f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a575455a3d2f4277bae898425ebbee06', 'request_id': 'req-6f97f64b-e610-4acc-8aef-85305cd1c83f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:36.876086 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6f97f64b-e610-4acc-8aef-85305cd1c83f admin admin] The token cache is being invalidate because project a575455a3d2f4277bae898425ebbee06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:15:36.876371 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6f97f64b-e610-4acc-8aef-85305cd1c83f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:36.877063 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6f97f64b-e610-4acc-8aef-85305cd1c83f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:36.878814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 964/1922] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:15:36 2022] DELETE /identity/v3/projects/a575455a3d2f4277bae898425ebbee06 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:15:36.884482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5473510-9185-4fff-b547-d93fe1318838 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:36.885669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5473510-9185-4fff-b547-d93fe1318838 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:36.886011 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5473510-9185-4fff-b547-d93fe1318838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:36.926668 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e5473510-9185-4fff-b547-d93fe1318838 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:36.928531 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e5473510-9185-4fff-b547-d93fe1318838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:36.936568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3dedc-8bb1-48c2-a46d-a7f13616ddfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:36.937244 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3dedc-8bb1-48c2-a46d-a7f13616ddfe admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:36.937690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3dedc-8bb1-48c2-a46d-a7f13616ddfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:36.939884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3dedc-8bb1-48c2-a46d-a7f13616ddfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:36.939884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b3dedc-8bb1-48c2-a46d-a7f13616ddfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:36.940703 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b3dedc-8bb1-48c2-a46d-a7f13616ddfe admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:36.954401 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 959/1923] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:15:36.957656 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3dedc-8bb1-48c2-a46d-a7f13616ddfe admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:36.959557 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-55b3dedc-8bb1-48c2-a46d-a7f13616ddfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:36.976615 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3dedc-8bb1-48c2-a46d-a7f13616ddfe admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:36.983807 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5274d46a-737d-4cbf-8817-04e7a8ca06ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:36.984616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274d46a-737d-4cbf-8817-04e7a8ca06ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:36.984921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274d46a-737d-4cbf-8817-04e7a8ca06ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:36.985409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274d46a-737d-4cbf-8817-04e7a8ca06ce admin admin] PATH_INFO: `/v3/projects/9df03841d74a4020a5b917ef843c54f8/users/ffe222654333427e92f261b56f4e9c8f/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:36.988360 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5274d46a-737d-4cbf-8817-04e7a8ca06ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=9df03841d74a4020a5b917ef843c54f8, user_id=ffe222654333427e92f261b56f4e9c8f, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:36.991878 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5274d46a-737d-4cbf-8817-04e7a8ca06ce admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:37.007464 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 960/1924] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:15:36 2022] PUT /identity/v3/projects/9df03841d74a4020a5b917ef843c54f8/users/ffe222654333427e92f261b56f4e9c8f/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:15:37.013631 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 965/1925] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:15:37.020198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87f6d4-f37d-4897-a1ae-f43b0312a69f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:37.021792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87f6d4-f37d-4897-a1ae-f43b0312a69f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:37.021792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87f6d4-f37d-4897-a1ae-f43b0312a69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:37.022064 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef8302-76a8-4bcd-8146-97a97dfe5ce8 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:37.022204 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef8302-76a8-4bcd-8146-97a97dfe5ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:37.022966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef8302-76a8-4bcd-8146-97a97dfe5ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:37.055469 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-20ef8302-76a8-4bcd-8146-97a97dfe5ce8 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:37.058503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-20ef8302-76a8-4bcd-8146-97a97dfe5ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:37.078024 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0c87f6d4-f37d-4897-a1ae-f43b0312a69f None None] MFA Rules not processed for user `ffe222654333427e92f261b56f4e9c8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:37.080410 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0c87f6d4-f37d-4897-a1ae-f43b0312a69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:37.184975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 966/1926] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:37.198553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef8302-76a8-4bcd-8146-97a97dfe5ce8 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:37.200232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:37.207532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 961/1927] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:37 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:37.209258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef8302-76a8-4bcd-8146-97a97dfe5ce8 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:37.215605 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e99282-2a56-42d3-9f58-c2ff93af0d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:37.216364 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e99282-2a56-42d3-9f58-c2ff93af0d92 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:37.216561 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e99282-2a56-42d3-9f58-c2ff93af0d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:37.216831 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e99282-2a56-42d3-9f58-c2ff93af0d92 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:37.217489 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e99282-2a56-42d3-9f58-c2ff93af0d92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:15:37.217793 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e99282-2a56-42d3-9f58-c2ff93af0d92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:37.219610 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e99282-2a56-42d3-9f58-c2ff93af0d92 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:37.221036 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87f6d4-f37d-4897-a1ae-f43b0312a69f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:37.224971 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0c87f6d4-f37d-4897-a1ae-f43b0312a69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:37.240428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 967/1928] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 17:15:37.243978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e99282-2a56-42d3-9f58-c2ff93af0d92 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:37.267072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e99282-2a56-42d3-9f58-c2ff93af0d92 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:37.273418 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add41975-ac0f-4cc1-888f-dea3d62b35ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:37.274282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add41975-ac0f-4cc1-888f-dea3d62b35ed admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:37.275170 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add41975-ac0f-4cc1-888f-dea3d62b35ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:37.275625 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add41975-ac0f-4cc1-888f-dea3d62b35ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:37.276496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add41975-ac0f-4cc1-888f-dea3d62b35ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:37.278493 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add41975-ac0f-4cc1-888f-dea3d62b35ed admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:37.300074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:37.345358 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87f6d4-f37d-4897-a1ae-f43b0312a69f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:37.349781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eadf49-f19b-4885-a303-cf04f642a0be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:37.350473 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eadf49-f19b-4885-a303-cf04f642a0be service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:37.350754 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eadf49-f19b-4885-a303-cf04f642a0be service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:37.351026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eadf49-f19b-4885-a303-cf04f642a0be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:37.363142 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eadf49-f19b-4885-a303-cf04f642a0be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:37.364600 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eadf49-f19b-4885-a303-cf04f642a0be service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:37.375933 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 962/1929] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:15:37 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:15:37.403845 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 968/1930] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:15:37 2022] POST /identity/v3/projects => generated 371 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:15:37.406058 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eadf49-f19b-4885-a303-cf04f642a0be service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:37.429727 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eadf49-f19b-4885-a303-cf04f642a0be service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:37.506687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c985f91-2ffb-4e2a-aaf0-7108885ec07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:37.507388 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c985f91-2ffb-4e2a-aaf0-7108885ec07e admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:37.507644 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c985f91-2ffb-4e2a-aaf0-7108885ec07e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:37.507827 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c985f91-2ffb-4e2a-aaf0-7108885ec07e admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:37.508455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c985f91-2ffb-4e2a-aaf0-7108885ec07e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:37.510617 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c985f91-2ffb-4e2a-aaf0-7108885ec07e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:37.626308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 963/1931] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:15:37 2022] POST /identity/v3/users => generated 286 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:15:37.631500 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add41975-ac0f-4cc1-888f-dea3d62b35ed admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:37.640785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c985f91-2ffb-4e2a-aaf0-7108885ec07e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:37.642606 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6c985f91-2ffb-4e2a-aaf0-7108885ec07e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:37.653571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add41975-ac0f-4cc1-888f-dea3d62b35ed admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:37.661456 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264dd86-1068-46c1-90bb-055a92758012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:37.662458 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264dd86-1068-46c1-90bb-055a92758012 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:37.662950 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264dd86-1068-46c1-90bb-055a92758012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:37.663440 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264dd86-1068-46c1-90bb-055a92758012 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:37.664635 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4264dd86-1068-46c1-90bb-055a92758012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:37.667656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4264dd86-1068-46c1-90bb-055a92758012 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:37.682801 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 969/1932] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:15:37.687082 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264dd86-1068-46c1-90bb-055a92758012 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:37.698196 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264dd86-1068-46c1-90bb-055a92758012 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:37.704481 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115e406-3519-46b9-9269-af8cc888e73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:37.705385 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115e406-3519-46b9-9269-af8cc888e73c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:37.705990 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115e406-3519-46b9-9269-af8cc888e73c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:37.706610 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115e406-3519-46b9-9269-af8cc888e73c admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:37.707567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1115e406-3519-46b9-9269-af8cc888e73c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:37.709951 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1115e406-3519-46b9-9269-af8cc888e73c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:37.722897 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 970/1933] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:15:37.727449 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115e406-3519-46b9-9269-af8cc888e73c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:37.736311 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c985f91-2ffb-4e2a-aaf0-7108885ec07e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:37.739478 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115e406-3519-46b9-9269-af8cc888e73c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:37.740445 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70375664-9e2f-40fc-8109-3c5e3cd88ea8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:37.741130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70375664-9e2f-40fc-8109-3c5e3cd88ea8 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:37.741315 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70375664-9e2f-40fc-8109-3c5e3cd88ea8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:37.741526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70375664-9e2f-40fc-8109-3c5e3cd88ea8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:37.744335 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-70375664-9e2f-40fc-8109-3c5e3cd88ea8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:37.746244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8bc534-8be4-4838-a782-37bba9c36d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:37.752669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bc534-8be4-4838-a782-37bba9c36d22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:37.752669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bc534-8be4-4838-a782-37bba9c36d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:37.752669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bc534-8be4-4838-a782-37bba9c36d22 admin admin] PATH_INFO: `/v3/projects/3940b616cbc448ab9d6eb31560814e90/users/afbc55d3fcdc4782b7e0cafbf38de5b6/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:37.775366 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8bc534-8be4-4838-a782-37bba9c36d22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3940b616cbc448ab9d6eb31560814e90, user_id=afbc55d3fcdc4782b7e0cafbf38de5b6, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:37.780786 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8bc534-8be4-4838-a782-37bba9c36d22 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:37.818990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:37.836228 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70375664-9e2f-40fc-8109-3c5e3cd88ea8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:37.838661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70375664-9e2f-40fc-8109-3c5e3cd88ea8 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:37.848780 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 964/1934] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:15:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:15:38.016863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 971/1935] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:15:37 2022] PUT /identity/v3/projects/3940b616cbc448ab9d6eb31560814e90/users/afbc55d3fcdc4782b7e0cafbf38de5b6/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:15:38.028359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7523e-7ff3-41c1-9683-719a74b9b59f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:38.028626 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7523e-7ff3-41c1-9683-719a74b9b59f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:38.028929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7523e-7ff3-41c1-9683-719a74b9b59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:38.077833 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-aef7523e-7ff3-41c1-9683-719a74b9b59f None None] MFA Rules not processed for user `afbc55d3fcdc4782b7e0cafbf38de5b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:38.084564 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-aef7523e-7ff3-41c1-9683-719a74b9b59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:38.190804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 965/1936] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:38.208077 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8bc534-8be4-4838-a782-37bba9c36d22 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:38.210584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2e8bc534-8be4-4838-a782-37bba9c36d22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:38.330734 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8bc534-8be4-4838-a782-37bba9c36d22 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:38.336694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4c7b1-338d-4d01-ace9-f947614cf936 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10f4f54890624da29ff217081445b6bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:38.339251 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4c7b1-338d-4d01-ace9-f947614cf936 service glance] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:38.339705 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4c7b1-338d-4d01-ace9-f947614cf936 service glance] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:38.340462 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4c7b1-338d-4d01-ace9-f947614cf936 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:38.360998 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a4c7b1-338d-4d01-ace9-f947614cf936 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:38.364327 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a4c7b1-338d-4d01-ace9-f947614cf936 service glance] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:38.391663 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 972/1937] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:15:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:15:47.659444 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef7523e-7ff3-41c1-9683-719a74b9b59f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:47.734203 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef7523e-7ff3-41c1-9683-719a74b9b59f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:47.741415 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb682e96-71c4-4fb1-a158-380e615d0970 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:47.742576 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb682e96-71c4-4fb1-a158-380e615d0970 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:47.743590 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb682e96-71c4-4fb1-a158-380e615d0970 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:47.744073 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb682e96-71c4-4fb1-a158-380e615d0970 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:47.747365 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fb682e96-71c4-4fb1-a158-380e615d0970 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:47.849700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb682e96-71c4-4fb1-a158-380e615d0970 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:47.851375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb682e96-71c4-4fb1-a158-380e615d0970 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:47.870173 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 966/1938] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:15:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:15:48.374067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=2,keystone:SELECT=57 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:53.621155 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4c7b1-338d-4d01-ace9-f947614cf936 service glance] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:53.709500 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4c7b1-338d-4d01-ace9-f947614cf936 service glance] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:53.717219 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20015054-6a05-43e6-98f6-d5e703022add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:53.717861 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20015054-6a05-43e6-98f6-d5e703022add admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:53.718131 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20015054-6a05-43e6-98f6-d5e703022add admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:53.718468 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20015054-6a05-43e6-98f6-d5e703022add admin admin] PATH_INFO: `/v3/users/5b00585340bf4ff380911677605160bd` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:53.719744 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20015054-6a05-43e6-98f6-d5e703022add admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b00585340bf4ff380911677605160bd)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:53.721854 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20015054-6a05-43e6-98f6-d5e703022add admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:53.966761 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-20015054-6a05-43e6-98f6-d5e703022add admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b00585340bf4ff380911677605160bd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:54.080502 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-20015054-6a05-43e6-98f6-d5e703022add admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b00585340bf4ff380911677605160bd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:54.101018 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-20015054-6a05-43e6-98f6-d5e703022add admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b00585340bf4ff380911677605160bd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:54.128669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-20015054-6a05-43e6-98f6-d5e703022add admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b00585340bf4ff380911677605160bd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:54.129776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-20015054-6a05-43e6-98f6-d5e703022add admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b00585340bf4ff380911677605160bd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:54.139006 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 973/1939] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:15:53 2022] DELETE /identity/v3/users/5b00585340bf4ff380911677605160bd => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:54.145184 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04967737-ff30-41d7-975f-71ef2e3b8a79 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:54.146106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04967737-ff30-41d7-975f-71ef2e3b8a79 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:54.146198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04967737-ff30-41d7-975f-71ef2e3b8a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:54.192488 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20015054-6a05-43e6-98f6-d5e703022add admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:54.196440 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-20015054-6a05-43e6-98f6-d5e703022add admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:54.201356 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-04967737-ff30-41d7-975f-71ef2e3b8a79 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:54.205212 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-04967737-ff30-41d7-975f-71ef2e3b8a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:54.259650 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:54.340987 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20015054-6a05-43e6-98f6-d5e703022add admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:54.356584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c167b9e-2746-43d9-a270-342d482f8e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:54.358593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c167b9e-2746-43d9-a270-342d482f8e81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:54.359102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c167b9e-2746-43d9-a270-342d482f8e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:54.359607 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c167b9e-2746-43d9-a270-342d482f8e81 admin admin] PATH_INFO: `/v3/users/5bef640e14af4d1cadce2e17296be286` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:54.362586 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c167b9e-2746-43d9-a270-342d482f8e81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bef640e14af4d1cadce2e17296be286)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:54.363007 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 967/1940] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:54.375425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c167b9e-2746-43d9-a270-342d482f8e81 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:54.383738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04967737-ff30-41d7-975f-71ef2e3b8a79 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:54.386064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-04967737-ff30-41d7-975f-71ef2e3b8a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:54.438814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:54.486651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04967737-ff30-41d7-975f-71ef2e3b8a79 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:54.493568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99410ef-04cb-4cc5-b849-732c10c0f16c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:54.494226 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99410ef-04cb-4cc5-b849-732c10c0f16c service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:54.494584 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99410ef-04cb-4cc5-b849-732c10c0f16c service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:54.494857 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99410ef-04cb-4cc5-b849-732c10c0f16c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:54.507738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99410ef-04cb-4cc5-b849-732c10c0f16c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:54.517122 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99410ef-04cb-4cc5-b849-732c10c0f16c service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:54.534613 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 968/1941] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:15:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:15:54.634443 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4c167b9e-2746-43d9-a270-342d482f8e81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bef640e14af4d1cadce2e17296be286'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:54.728882 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4c167b9e-2746-43d9-a270-342d482f8e81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bef640e14af4d1cadce2e17296be286'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:54.746724 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4c167b9e-2746-43d9-a270-342d482f8e81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bef640e14af4d1cadce2e17296be286'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:54.785043 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4c167b9e-2746-43d9-a270-342d482f8e81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bef640e14af4d1cadce2e17296be286'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:54.786169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4c167b9e-2746-43d9-a270-342d482f8e81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bef640e14af4d1cadce2e17296be286'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:54.806115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 974/1942] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:15:54 2022] DELETE /identity/v3/users/5bef640e14af4d1cadce2e17296be286 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:54.814610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e35dc-07d0-4610-a9c2-59fb7b08fc8a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:54.816326 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e35dc-07d0-4610-a9c2-59fb7b08fc8a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:54.816822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e35dc-07d0-4610-a9c2-59fb7b08fc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:54.854116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8a7e35dc-07d0-4610-a9c2-59fb7b08fc8a None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:54.856245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8a7e35dc-07d0-4610-a9c2-59fb7b08fc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:54.993879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 969/1943] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:55.007856 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c167b9e-2746-43d9-a270-342d482f8e81 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:55.011062 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4c167b9e-2746-43d9-a270-342d482f8e81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:55.144719 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c167b9e-2746-43d9-a270-342d482f8e81 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:55.152137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f0ead-94d9-4d65-8d44-a21737bee198 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:55.152137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f0ead-94d9-4d65-8d44-a21737bee198 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:55.152137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f0ead-94d9-4d65-8d44-a21737bee198 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:55.152137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f0ead-94d9-4d65-8d44-a21737bee198 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:55.172219 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f0ead-94d9-4d65-8d44-a21737bee198 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:55.174090 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f0ead-94d9-4d65-8d44-a21737bee198 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:55.187667 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 975/1944] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:15:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:15:55.666633 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e35dc-07d0-4610-a9c2-59fb7b08fc8a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:55.668886 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8a7e35dc-07d0-4610-a9c2-59fb7b08fc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:55.687404 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e35dc-07d0-4610-a9c2-59fb7b08fc8a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:55.694322 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e95fb0-cefe-447e-8bd2-6621a869a127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:55.695231 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e95fb0-cefe-447e-8bd2-6621a869a127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:55.695594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e95fb0-cefe-447e-8bd2-6621a869a127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:55.696092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e95fb0-cefe-447e-8bd2-6621a869a127 admin admin] PATH_INFO: `/v3/projects/4f89a6525d6742fd8b62dd380dc17418` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:55.698531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e95fb0-cefe-447e-8bd2-6621a869a127 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f89a6525d6742fd8b62dd380dc17418)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:55.704135 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e95fb0-cefe-447e-8bd2-6621a869a127 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:55.789212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f0ead-94d9-4d65-8d44-a21737bee198 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:55.792450 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a75f0ead-94d9-4d65-8d44-a21737bee198 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:55.837919 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f0ead-94d9-4d65-8d44-a21737bee198 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:55.859291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13298f34-d330-4807-9d4a-7ea452502705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:55.864407 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13298f34-d330-4807-9d4a-7ea452502705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:55.864407 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13298f34-d330-4807-9d4a-7ea452502705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:55.864407 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13298f34-d330-4807-9d4a-7ea452502705 admin admin] PATH_INFO: `/v3/projects/26a9d512d3be400d89d90bce5d3e62fc` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:55.865607 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13298f34-d330-4807-9d4a-7ea452502705 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26a9d512d3be400d89d90bce5d3e62fc)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:55.873298 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13298f34-d330-4807-9d4a-7ea452502705 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:55.894270 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-96e95fb0-cefe-447e-8bd2-6621a869a127 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f89a6525d6742fd8b62dd380dc17418', 'request_id': 'req-96e95fb0-cefe-447e-8bd2-6621a869a127'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:55.985657 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-96e95fb0-cefe-447e-8bd2-6621a869a127 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f89a6525d6742fd8b62dd380dc17418', 'request_id': 'req-96e95fb0-cefe-447e-8bd2-6621a869a127'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:56.003091 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-96e95fb0-cefe-447e-8bd2-6621a869a127 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f89a6525d6742fd8b62dd380dc17418', 'request_id': 'req-96e95fb0-cefe-447e-8bd2-6621a869a127'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:56.018489 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:56.040517 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-96e95fb0-cefe-447e-8bd2-6621a869a127 admin admin] The token cache is being invalidate because project 4f89a6525d6742fd8b62dd380dc17418 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:15:56.042301 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-96e95fb0-cefe-447e-8bd2-6621a869a127 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:56.043968 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-96e95fb0-cefe-447e-8bd2-6621a869a127 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:56.048054 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 970/1945] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:15:55 2022] DELETE /identity/v3/projects/4f89a6525d6742fd8b62dd380dc17418 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:56.054181 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e95fb0-cefe-447e-8bd2-6621a869a127 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:56.060719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-96e95fb0-cefe-447e-8bd2-6621a869a127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:56.067855 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-13298f34-d330-4807-9d4a-7ea452502705 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26a9d512d3be400d89d90bce5d3e62fc', 'request_id': 'req-13298f34-d330-4807-9d4a-7ea452502705'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:56.080586 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-13298f34-d330-4807-9d4a-7ea452502705 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26a9d512d3be400d89d90bce5d3e62fc', 'request_id': 'req-13298f34-d330-4807-9d4a-7ea452502705'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:56.151809 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-13298f34-d330-4807-9d4a-7ea452502705 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26a9d512d3be400d89d90bce5d3e62fc', 'request_id': 'req-13298f34-d330-4807-9d4a-7ea452502705'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:56.186536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e95fb0-cefe-447e-8bd2-6621a869a127 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:56.221671 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-13298f34-d330-4807-9d4a-7ea452502705 admin admin] The token cache is being invalidate because project 26a9d512d3be400d89d90bce5d3e62fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:15:56.222077 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-13298f34-d330-4807-9d4a-7ea452502705 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:56.222856 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-13298f34-d330-4807-9d4a-7ea452502705 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:56.226633 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 976/1946] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:15:55 2022] DELETE /identity/v3/projects/26a9d512d3be400d89d90bce5d3e62fc => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:15:56.232445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13298f34-d330-4807-9d4a-7ea452502705 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:56.234565 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-13298f34-d330-4807-9d4a-7ea452502705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:56.287221 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:56.353574 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5997ce4-4637-4a50-84c7-b7f6a51a5364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:56.354823 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5997ce4-4637-4a50-84c7-b7f6a51a5364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:56.355307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5997ce4-4637-4a50-84c7-b7f6a51a5364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:56.355762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5997ce4-4637-4a50-84c7-b7f6a51a5364 admin admin] PATH_INFO: `/v3/users/ffe222654333427e92f261b56f4e9c8f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:56.362845 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5997ce4-4637-4a50-84c7-b7f6a51a5364 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffe222654333427e92f261b56f4e9c8f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:56.366734 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5997ce4-4637-4a50-84c7-b7f6a51a5364 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:56.391266 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13298f34-d330-4807-9d4a-7ea452502705 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:56.402269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b2d07-1a09-49aa-9b08-d0fe976a007f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:56.402748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b2d07-1a09-49aa-9b08-d0fe976a007f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:56.403029 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b2d07-1a09-49aa-9b08-d0fe976a007f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:56.403295 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b2d07-1a09-49aa-9b08-d0fe976a007f admin admin] PATH_INFO: `/v3/users/ec32421a4fd34906be26af4a03fa49d4` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:56.404615 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b2d07-1a09-49aa-9b08-d0fe976a007f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec32421a4fd34906be26af4a03fa49d4)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:56.406685 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b2d07-1a09-49aa-9b08-d0fe976a007f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:56.660352 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2f3b2d07-1a09-49aa-9b08-d0fe976a007f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec32421a4fd34906be26af4a03fa49d4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:56.676838 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c5997ce4-4637-4a50-84c7-b7f6a51a5364 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffe222654333427e92f261b56f4e9c8f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:56.749364 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2f3b2d07-1a09-49aa-9b08-d0fe976a007f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec32421a4fd34906be26af4a03fa49d4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:56.771499 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2f3b2d07-1a09-49aa-9b08-d0fe976a007f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec32421a4fd34906be26af4a03fa49d4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:56.795517 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2f3b2d07-1a09-49aa-9b08-d0fe976a007f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec32421a4fd34906be26af4a03fa49d4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:56.797626 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2f3b2d07-1a09-49aa-9b08-d0fe976a007f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec32421a4fd34906be26af4a03fa49d4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:56.809202 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c5997ce4-4637-4a50-84c7-b7f6a51a5364 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffe222654333427e92f261b56f4e9c8f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:56.811745 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 977/1947] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:15:56 2022] DELETE /identity/v3/users/ec32421a4fd34906be26af4a03fa49d4 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:56.846917 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c5997ce4-4637-4a50-84c7-b7f6a51a5364 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffe222654333427e92f261b56f4e9c8f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:56.848223 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c5997ce4-4637-4a50-84c7-b7f6a51a5364 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffe222654333427e92f261b56f4e9c8f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:56.849510 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c5997ce4-4637-4a50-84c7-b7f6a51a5364 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffe222654333427e92f261b56f4e9c8f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:56.872685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 971/1948] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:15:56 2022] DELETE /identity/v3/users/ffe222654333427e92f261b56f4e9c8f => generated 0 bytes in 819 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:57.739077 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b2d07-1a09-49aa-9b08-d0fe976a007f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:57.741124 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2f3b2d07-1a09-49aa-9b08-d0fe976a007f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:57.850589 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b2d07-1a09-49aa-9b08-d0fe976a007f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:57.887079 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5997ce4-4637-4a50-84c7-b7f6a51a5364 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:57.889120 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c5997ce4-4637-4a50-84c7-b7f6a51a5364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:57.942702 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:57.965440 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:57.978114 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adeb0f2-14aa-413e-99b8-ca24d926f606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:57.978806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adeb0f2-14aa-413e-99b8-ca24d926f606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:57.979084 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adeb0f2-14aa-413e-99b8-ca24d926f606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:57.979356 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adeb0f2-14aa-413e-99b8-ca24d926f606 admin admin] PATH_INFO: `/v3/projects/9df03841d74a4020a5b917ef843c54f8` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:57.981262 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adeb0f2-14aa-413e-99b8-ca24d926f606 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9df03841d74a4020a5b917ef843c54f8)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:57.983293 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adeb0f2-14aa-413e-99b8-ca24d926f606 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:58.020778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5997ce4-4637-4a50-84c7-b7f6a51a5364 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:58.026940 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb230c22-8f0e-4f76-b644-b4e9b4b07b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:58.027670 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb230c22-8f0e-4f76-b644-b4e9b4b07b19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:58.027971 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb230c22-8f0e-4f76-b644-b4e9b4b07b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:58.033001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb230c22-8f0e-4f76-b644-b4e9b4b07b19 admin admin] PATH_INFO: `/v3/projects/1593aaf958404b058550332d32e2d0af` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:58.033001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb230c22-8f0e-4f76-b644-b4e9b4b07b19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1593aaf958404b058550332d32e2d0af)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:58.033001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb230c22-8f0e-4f76-b644-b4e9b4b07b19 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:58.098729 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4adeb0f2-14aa-413e-99b8-ca24d926f606 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9df03841d74a4020a5b917ef843c54f8', 'request_id': 'req-4adeb0f2-14aa-413e-99b8-ca24d926f606'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:58.108483 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4adeb0f2-14aa-413e-99b8-ca24d926f606 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9df03841d74a4020a5b917ef843c54f8', 'request_id': 'req-4adeb0f2-14aa-413e-99b8-ca24d926f606'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:58.192619 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-eb230c22-8f0e-4f76-b644-b4e9b4b07b19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1593aaf958404b058550332d32e2d0af', 'request_id': 'req-eb230c22-8f0e-4f76-b644-b4e9b4b07b19'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:58.246415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4adeb0f2-14aa-413e-99b8-ca24d926f606 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9df03841d74a4020a5b917ef843c54f8', 'request_id': 'req-4adeb0f2-14aa-413e-99b8-ca24d926f606'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:58.260719 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4adeb0f2-14aa-413e-99b8-ca24d926f606 admin admin] The token cache is being invalidate because project 9df03841d74a4020a5b917ef843c54f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:15:58.261233 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4adeb0f2-14aa-413e-99b8-ca24d926f606 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:58.262445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4adeb0f2-14aa-413e-99b8-ca24d926f606 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:58.264037 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 978/1949] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:15:57 2022] DELETE /identity/v3/projects/9df03841d74a4020a5b917ef843c54f8 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:15:58.266156 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8179f-518d-42f1-a321-1868483db468 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:58.266682 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8179f-518d-42f1-a321-1868483db468 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:58.267169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8179f-518d-42f1-a321-1868483db468 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:58.269484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 979/1950] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 10 17:15:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:15:58.271010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409884a-400e-4023-8d76-54b12db0988a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:58.271358 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409884a-400e-4023-8d76-54b12db0988a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:58.271694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409884a-400e-4023-8d76-54b12db0988a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:58.303209 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0409884a-400e-4023-8d76-54b12db0988a None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:58.304446 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0409884a-400e-4023-8d76-54b12db0988a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:58.377467 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-eb230c22-8f0e-4f76-b644-b4e9b4b07b19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1593aaf958404b058550332d32e2d0af', 'request_id': 'req-eb230c22-8f0e-4f76-b644-b4e9b4b07b19'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:58.379168 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 980/1951] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:58.381195 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd88f075-d15e-4bf2-b294-149481af7e4b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:58.381613 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd88f075-d15e-4bf2-b294-149481af7e4b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:58.382007 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd88f075-d15e-4bf2-b294-149481af7e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:58.387181 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-eb230c22-8f0e-4f76-b644-b4e9b4b07b19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1593aaf958404b058550332d32e2d0af', 'request_id': 'req-eb230c22-8f0e-4f76-b644-b4e9b4b07b19'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:58.423268 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bd88f075-d15e-4bf2-b294-149481af7e4b None None] MFA Rules not processed for user `8cd39d9a583a4d73a5654017be106019`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:58.426200 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bd88f075-d15e-4bf2-b294-149481af7e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:58.427999 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-eb230c22-8f0e-4f76-b644-b4e9b4b07b19 admin admin] The token cache is being invalidate because project 1593aaf958404b058550332d32e2d0af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:15:58.428848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-eb230c22-8f0e-4f76-b644-b4e9b4b07b19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:58.430424 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-eb230c22-8f0e-4f76-b644-b4e9b4b07b19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:58.441354 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 972/1952] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:15:57 2022] DELETE /identity/v3/projects/1593aaf958404b058550332d32e2d0af => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:15:58.450509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb230c22-8f0e-4f76-b644-b4e9b4b07b19 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:58.450509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-eb230c22-8f0e-4f76-b644-b4e9b4b07b19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:58.473613 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb230c22-8f0e-4f76-b644-b4e9b4b07b19 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:58.479331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ee7c0-a888-48ee-ac0d-2118f7c52d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:58.479973 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ee7c0-a888-48ee-ac0d-2118f7c52d94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:58.480258 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ee7c0-a888-48ee-ac0d-2118f7c52d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:58.480534 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ee7c0-a888-48ee-ac0d-2118f7c52d94 admin admin] PATH_INFO: `/v3/users/06cb7a8f70ab409b86c669620df4a651` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:58.481721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ee7c0-a888-48ee-ac0d-2118f7c52d94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06cb7a8f70ab409b86c669620df4a651)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:58.483714 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ee7c0-a888-48ee-ac0d-2118f7c52d94 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:58.527660 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 981/1953] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 10 17:15:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:58.529916 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba81ef-0c8b-4830-b3bb-e32c9e89921a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:58.533487 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba81ef-0c8b-4830-b3bb-e32c9e89921a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:58.533487 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba81ef-0c8b-4830-b3bb-e32c9e89921a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:58.554143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d9ba81ef-0c8b-4830-b3bb-e32c9e89921a None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:58.556122 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d9ba81ef-0c8b-4830-b3bb-e32c9e89921a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:58.565214 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 982/1954] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:58.566866 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed75024-16f4-4ef7-b9e8-c0dcf4116a4b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:58.567277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed75024-16f4-4ef7-b9e8-c0dcf4116a4b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:58.567641 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed75024-16f4-4ef7-b9e8-c0dcf4116a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:58.595554 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bed75024-16f4-4ef7-b9e8-c0dcf4116a4b None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:58.596809 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bed75024-16f4-4ef7-b9e8-c0dcf4116a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:58.606986 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 983/1955] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:58.608656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed75024-16f4-4ef7-b9e8-c0dcf4116a4b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:58.610703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bed75024-16f4-4ef7-b9e8-c0dcf4116a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:58.631260 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:58.636392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed75024-16f4-4ef7-b9e8-c0dcf4116a4b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:58.641849 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac18dd-aecb-4a24-b0f7-e0be77993468 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cd39d9a583a4d73a5654017be106019', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:58.642614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac18dd-aecb-4a24-b0f7-e0be77993468 service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:58.643027 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac18dd-aecb-4a24-b0f7-e0be77993468 service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:58.643475 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac18dd-aecb-4a24-b0f7-e0be77993468 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:58.654209 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac18dd-aecb-4a24-b0f7-e0be77993468 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:58.656069 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac18dd-aecb-4a24-b0f7-e0be77993468 service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:58.667371 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 984/1956] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:15:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:15:58.668787 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac18dd-aecb-4a24-b0f7-e0be77993468 service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:58.684171 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac18dd-aecb-4a24-b0f7-e0be77993468 service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:58.691057 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7f833-634d-457d-9c38-a14144e0a6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:58.691792 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7f833-634d-457d-9c38-a14144e0a6a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:58.692082 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7f833-634d-457d-9c38-a14144e0a6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:58.692358 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7f833-634d-457d-9c38-a14144e0a6a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:58.693197 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7f833-634d-457d-9c38-a14144e0a6a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:15:58.693503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7f833-634d-457d-9c38-a14144e0a6a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:58.695348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7f833-634d-457d-9c38-a14144e0a6a6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:58.713129 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 985/1957] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:15:58.723607 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5f9e3-9aea-4a21-8b2a-f2361c254a1a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:58.723607 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5f9e3-9aea-4a21-8b2a-f2361c254a1a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:58.723607 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5f9e3-9aea-4a21-8b2a-f2361c254a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:58.735020 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-0d2ee7c0-a888-48ee-ac0d-2118f7c52d94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06cb7a8f70ab409b86c669620df4a651'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:58.742918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c4a5f9e3-9aea-4a21-8b2a-f2361c254a1a None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:58.744396 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c4a5f9e3-9aea-4a21-8b2a-f2361c254a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:58.753354 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 986/1958] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:58.754608 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5f9e3-9aea-4a21-8b2a-f2361c254a1a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:58.765931 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5f9e3-9aea-4a21-8b2a-f2361c254a1a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:58.771230 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f806230-b7db-4174-b84c-04d157dbfe4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:58.771862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f806230-b7db-4174-b84c-04d157dbfe4d admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:58.772129 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f806230-b7db-4174-b84c-04d157dbfe4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:58.772392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f806230-b7db-4174-b84c-04d157dbfe4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:58.773033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f806230-b7db-4174-b84c-04d157dbfe4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:58.775843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f806230-b7db-4174-b84c-04d157dbfe4d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:58.794735 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-0d2ee7c0-a888-48ee-ac0d-2118f7c52d94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06cb7a8f70ab409b86c669620df4a651'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:58.812088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-0d2ee7c0-a888-48ee-ac0d-2118f7c52d94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06cb7a8f70ab409b86c669620df4a651'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:58.812786 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-0d2ee7c0-a888-48ee-ac0d-2118f7c52d94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06cb7a8f70ab409b86c669620df4a651'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:58.813444 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-0d2ee7c0-a888-48ee-ac0d-2118f7c52d94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06cb7a8f70ab409b86c669620df4a651'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:58.828078 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 973/1959] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:15:58 2022] DELETE /identity/v3/users/06cb7a8f70ab409b86c669620df4a651 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:58.829022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ee7c0-a888-48ee-ac0d-2118f7c52d94 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:58.830659 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0d2ee7c0-a888-48ee-ac0d-2118f7c52d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:58.865572 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:58.877515 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 987/1960] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:15:58 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:15:58.883976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f806230-b7db-4174-b84c-04d157dbfe4d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:58.887233 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0f806230-b7db-4174-b84c-04d157dbfe4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:58.927868 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ee7c0-a888-48ee-ac0d-2118f7c52d94 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:59.030699 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f806230-b7db-4174-b84c-04d157dbfe4d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:59.045904 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcd823-748d-4f5f-88b0-e03e416ab9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:59.046900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcd823-748d-4f5f-88b0-e03e416ab9fe admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:59.047347 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcd823-748d-4f5f-88b0-e03e416ab9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:59.047627 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcd823-748d-4f5f-88b0-e03e416ab9fe admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:59.048610 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fcd823-748d-4f5f-88b0-e03e416ab9fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:59.055266 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fcd823-748d-4f5f-88b0-e03e416ab9fe admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:59.064823 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18134162-3487-4abe-8474-ca02bfc34364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:59.065926 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18134162-3487-4abe-8474-ca02bfc34364 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:59.066592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18134162-3487-4abe-8474-ca02bfc34364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:59.067216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18134162-3487-4abe-8474-ca02bfc34364 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:59.068547 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18134162-3487-4abe-8474-ca02bfc34364 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:15:59.070086 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18134162-3487-4abe-8474-ca02bfc34364 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:59.073503 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18134162-3487-4abe-8474-ca02bfc34364 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:59.090431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 974/1961] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:15:59.095246 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18134162-3487-4abe-8474-ca02bfc34364 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:59.110825 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18134162-3487-4abe-8474-ca02bfc34364 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:59.117799 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd110e68-f8f8-4f3f-9efd-fe8dbe610933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:59.119774 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd110e68-f8f8-4f3f-9efd-fe8dbe610933 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:59.120612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd110e68-f8f8-4f3f-9efd-fe8dbe610933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:59.121314 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd110e68-f8f8-4f3f-9efd-fe8dbe610933 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:59.122782 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd110e68-f8f8-4f3f-9efd-fe8dbe610933 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:59.125771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd110e68-f8f8-4f3f-9efd-fe8dbe610933 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:59.178910 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 988/1962] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:15:58 2022] POST /identity/v3/users => generated 292 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:15:59.183795 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcd823-748d-4f5f-88b0-e03e416ab9fe admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:59.193781 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcd823-748d-4f5f-88b0-e03e416ab9fe admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:59.201058 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60300628-40d5-4bef-9221-45ea8a86ff49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:59.202693 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60300628-40d5-4bef-9221-45ea8a86ff49 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:59.203279 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60300628-40d5-4bef-9221-45ea8a86ff49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:59.203637 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60300628-40d5-4bef-9221-45ea8a86ff49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:59.204612 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60300628-40d5-4bef-9221-45ea8a86ff49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:59.206842 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60300628-40d5-4bef-9221-45ea8a86ff49 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:59.222496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 989/1963] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:15:59.227111 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60300628-40d5-4bef-9221-45ea8a86ff49 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:59.238000 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60300628-40d5-4bef-9221-45ea8a86ff49 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:59.242930 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9e3e5-aae2-4f35-9db7-2d6c5cfd0bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:59.244252 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9e3e5-aae2-4f35-9db7-2d6c5cfd0bab admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:59.244865 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9e3e5-aae2-4f35-9db7-2d6c5cfd0bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:59.247885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9e3e5-aae2-4f35-9db7-2d6c5cfd0bab admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:59.247885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee9e3e5-aae2-4f35-9db7-2d6c5cfd0bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:59.247885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee9e3e5-aae2-4f35-9db7-2d6c5cfd0bab admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:59.266494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 990/1964] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:15:59.267942 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9e3e5-aae2-4f35-9db7-2d6c5cfd0bab admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:59.271077 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bee9e3e5-aae2-4f35-9db7-2d6c5cfd0bab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:59.287525 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 975/1965] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:15:59 2022] POST /identity/v3/projects => generated 373 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:15:59.289265 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd110e68-f8f8-4f3f-9efd-fe8dbe610933 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:59.296456 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9e3e5-aae2-4f35-9db7-2d6c5cfd0bab admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:59.382316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a1c15-1af9-4114-8f6d-a701e58fa60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:59.383010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a1c15-1af9-4114-8f6d-a701e58fa60b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:59.383287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a1c15-1af9-4114-8f6d-a701e58fa60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:59.384122 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a1c15-1af9-4114-8f6d-a701e58fa60b admin admin] PATH_INFO: `/v3/projects/8a7cd3e7d80845e89b398f0707097ffd` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:59.385694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86a1c15-1af9-4114-8f6d-a701e58fa60b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a7cd3e7d80845e89b398f0707097ffd)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:59.388024 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86a1c15-1af9-4114-8f6d-a701e58fa60b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:59.397289 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd110e68-f8f8-4f3f-9efd-fe8dbe610933 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:59.405336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232c10f-7e6c-4ce2-9e0b-aaae75e65b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:59.406055 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232c10f-7e6c-4ce2-9e0b-aaae75e65b12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:59.407482 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232c10f-7e6c-4ce2-9e0b-aaae75e65b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:59.407482 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232c10f-7e6c-4ce2-9e0b-aaae75e65b12 admin admin] PATH_INFO: `/v3/projects/d78379f2ae4b4aad8c69deed4fdfb65f/users/567aa4145ac844669b802bb06faf7abc/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:59.434080 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e232c10f-7e6c-4ce2-9e0b-aaae75e65b12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d78379f2ae4b4aad8c69deed4fdfb65f, user_id=567aa4145ac844669b802bb06faf7abc, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:59.437263 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e232c10f-7e6c-4ce2-9e0b-aaae75e65b12 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:59.533340 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:59.542514 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 976/1966] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:15:59 2022] PUT /identity/v3/projects/d78379f2ae4b4aad8c69deed4fdfb65f/users/567aa4145ac844669b802bb06faf7abc/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:59.556770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232c10f-7e6c-4ce2-9e0b-aaae75e65b12 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:59.562762 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b86a1c15-1af9-4114-8f6d-a701e58fa60b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a7cd3e7d80845e89b398f0707097ffd', 'request_id': 'req-b86a1c15-1af9-4114-8f6d-a701e58fa60b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:59.573420 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b86a1c15-1af9-4114-8f6d-a701e58fa60b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a7cd3e7d80845e89b398f0707097ffd', 'request_id': 'req-b86a1c15-1af9-4114-8f6d-a701e58fa60b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:59.600659 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:15:59.627554 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232c10f-7e6c-4ce2-9e0b-aaae75e65b12 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:59.649510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b86a1c15-1af9-4114-8f6d-a701e58fa60b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a7cd3e7d80845e89b398f0707097ffd', 'request_id': 'req-b86a1c15-1af9-4114-8f6d-a701e58fa60b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:59.664402 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b86a1c15-1af9-4114-8f6d-a701e58fa60b admin admin] The token cache is being invalidate because project 8a7cd3e7d80845e89b398f0707097ffd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:15:59.665202 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b86a1c15-1af9-4114-8f6d-a701e58fa60b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:59.666419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b86a1c15-1af9-4114-8f6d-a701e58fa60b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:15:59.669134 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 991/1967] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:15:59 2022] DELETE /identity/v3/projects/8a7cd3e7d80845e89b398f0707097ffd => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:15:59.673036 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2a59a-0508-4827-8d70-2ec00d26e768 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:59.673036 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2a59a-0508-4827-8d70-2ec00d26e768 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:59.673036 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2a59a-0508-4827-8d70-2ec00d26e768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:59.727903 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013289a9-8dde-47db-8c5b-9f2c8b3b3277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:59.728911 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013289a9-8dde-47db-8c5b-9f2c8b3b3277 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:59.729347 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013289a9-8dde-47db-8c5b-9f2c8b3b3277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:59.729579 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-25e2a59a-0508-4827-8d70-2ec00d26e768 None None] MFA Rules not processed for user `567aa4145ac844669b802bb06faf7abc`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:59.729838 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013289a9-8dde-47db-8c5b-9f2c8b3b3277 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:59.730918 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013289a9-8dde-47db-8c5b-9f2c8b3b3277 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:59.731601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-25e2a59a-0508-4827-8d70-2ec00d26e768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:59.734337 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013289a9-8dde-47db-8c5b-9f2c8b3b3277 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:59.812810 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 992/1968] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:59.814090 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbaa3fc-0f5e-4949-b8bc-3f991f3334e0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:59.814394 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbaa3fc-0f5e-4949-b8bc-3f991f3334e0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:59.814667 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbaa3fc-0f5e-4949-b8bc-3f991f3334e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:59.838493 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8fbaa3fc-0f5e-4949-b8bc-3f991f3334e0 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:59.840591 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8fbaa3fc-0f5e-4949-b8bc-3f991f3334e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:59.848754 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 977/1969] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:15:59 2022] POST /identity/v3/users => generated 287 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:15:59.850203 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013289a9-8dde-47db-8c5b-9f2c8b3b3277 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:59.852789 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-013289a9-8dde-47db-8c5b-9f2c8b3b3277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:59.855321 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 993/1970] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:59.856853 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbaa3fc-0f5e-4949-b8bc-3f991f3334e0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:59.859595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8fbaa3fc-0f5e-4949-b8bc-3f991f3334e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:59.873279 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013289a9-8dde-47db-8c5b-9f2c8b3b3277 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:59.879468 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbaa3fc-0f5e-4949-b8bc-3f991f3334e0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:59.880563 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01337c71-44a5-4e31-89ec-ee3ba994279c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:59.881547 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01337c71-44a5-4e31-89ec-ee3ba994279c admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:59.881994 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01337c71-44a5-4e31-89ec-ee3ba994279c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:59.882504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01337c71-44a5-4e31-89ec-ee3ba994279c admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:59.884068 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01337c71-44a5-4e31-89ec-ee3ba994279c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:59.887695 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01337c71-44a5-4e31-89ec-ee3ba994279c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:59.891414 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e8909-0f9a-4712-a60d-0e1c0c8ded44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:59.891414 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e8909-0f9a-4712-a60d-0e1c0c8ded44 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:59.891414 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e8909-0f9a-4712-a60d-0e1c0c8ded44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:59.891414 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e8909-0f9a-4712-a60d-0e1c0c8ded44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:59.892034 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e8909-0f9a-4712-a60d-0e1c0c8ded44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:59.894921 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e8909-0f9a-4712-a60d-0e1c0c8ded44 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:59.912260 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 994/1971] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:15:59.918298 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10d176-c0ae-4028-973a-e96ed46ded7e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:59.918771 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10d176-c0ae-4028-973a-e96ed46ded7e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:59.919217 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10d176-c0ae-4028-973a-e96ed46ded7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:59.942851 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ec10d176-c0ae-4028-973a-e96ed46ded7e None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:15:59.944251 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ec10d176-c0ae-4028-973a-e96ed46ded7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:15:59.953596 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 995/1972] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:15:59.954550 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10d176-c0ae-4028-973a-e96ed46ded7e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:59.965324 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10d176-c0ae-4028-973a-e96ed46ded7e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:15:59.970249 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b11ad-f72c-4421-9b89-9f92c78872ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:15:59.970895 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b11ad-f72c-4421-9b89-9f92c78872ed admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:15:59.971166 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b11ad-f72c-4421-9b89-9f92c78872ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:15:59.972128 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b11ad-f72c-4421-9b89-9f92c78872ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:15:59.972128 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b11ad-f72c-4421-9b89-9f92c78872ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:15:59.973697 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b11ad-f72c-4421-9b89-9f92c78872ed admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:15:59.985637 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 996/1973] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:15:59.986548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b11ad-f72c-4421-9b89-9f92c78872ed admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:15:59.997969 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 978/1974] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:15:59 2022] POST /identity/v3/projects => generated 381 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:15:59.999342 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01337c71-44a5-4e31-89ec-ee3ba994279c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:00.002697 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b11ad-f72c-4421-9b89-9f92c78872ed admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:00.079025 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f4b04-b228-4be1-8054-93c9a6b91e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:00.079983 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f4b04-b228-4be1-8054-93c9a6b91e4a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:00.080431 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f4b04-b228-4be1-8054-93c9a6b91e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:00.080880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f4b04-b228-4be1-8054-93c9a6b91e4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:00.081963 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9f4b04-b228-4be1-8054-93c9a6b91e4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:16:00.082485 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9f4b04-b228-4be1-8054-93c9a6b91e4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:00.085061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9f4b04-b228-4be1-8054-93c9a6b91e4a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:00.090494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:00.100584 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01337c71-44a5-4e31-89ec-ee3ba994279c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:00.107063 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 997/1975] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:16:00.107331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd3111-e7ca-482e-9332-6a30ec077209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:00.108138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f4b04-b228-4be1-8054-93c9a6b91e4a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:00.108398 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd3111-e7ca-482e-9332-6a30ec077209 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:00.108872 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd3111-e7ca-482e-9332-6a30ec077209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:00.109551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd3111-e7ca-482e-9332-6a30ec077209 admin admin] PATH_INFO: `/v3/projects/201437c0acd2463cb2f09424dfc6f085/users/210be08699fb4653a32b6538aae2ebff/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:00.128076 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f4b04-b228-4be1-8054-93c9a6b91e4a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:00.133035 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd3111-e7ca-482e-9332-6a30ec077209 admin admin] RBAC: Authorizing `identity:create_grant(project_id=201437c0acd2463cb2f09424dfc6f085, user_id=210be08699fb4653a32b6538aae2ebff, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:00.134577 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87832bed-c0a2-45b4-9634-d350f2538e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:00.135308 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87832bed-c0a2-45b4-9634-d350f2538e6f admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:00.135788 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87832bed-c0a2-45b4-9634-d350f2538e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:00.136209 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87832bed-c0a2-45b4-9634-d350f2538e6f admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:00.137141 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87832bed-c0a2-45b4-9634-d350f2538e6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:00.139108 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd3111-e7ca-482e-9332-6a30ec077209 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:00.139281 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87832bed-c0a2-45b4-9634-d350f2538e6f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:00.247308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 979/1976] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:15:59 2022] PUT /identity/v3/projects/201437c0acd2463cb2f09424dfc6f085/users/210be08699fb4653a32b6538aae2ebff/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:16:00.247308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd3111-e7ca-482e-9332-6a30ec077209 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:00.273593 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:00.282590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 998/1977] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:16:00 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:16:00.282590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88deeb65-538d-493b-b58b-c62001233422 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:00.282590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88deeb65-538d-493b-b58b-c62001233422 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:00.282590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88deeb65-538d-493b-b58b-c62001233422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:00.335325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-88deeb65-538d-493b-b58b-c62001233422 None None] MFA Rules not processed for user `210be08699fb4653a32b6538aae2ebff`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:00.337530 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-88deeb65-538d-493b-b58b-c62001233422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:00.340861 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd3111-e7ca-482e-9332-6a30ec077209 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:00.350032 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5c456-8006-4f1e-afb8-035c4a9b107e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:00.351273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5c456-8006-4f1e-afb8-035c4a9b107e admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:00.351909 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5c456-8006-4f1e-afb8-035c4a9b107e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:00.352673 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5c456-8006-4f1e-afb8-035c4a9b107e admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:00.353936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5c456-8006-4f1e-afb8-035c4a9b107e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:00.359172 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5c456-8006-4f1e-afb8-035c4a9b107e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:00.449867 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 999/1978] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:16:00.451060 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88deeb65-538d-493b-b58b-c62001233422 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:00.468696 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88deeb65-538d-493b-b58b-c62001233422 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:00.478408 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdff41-b94d-467c-8c49-8365b51b1f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:00.479680 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdff41-b94d-467c-8c49-8365b51b1f11 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:00.480181 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdff41-b94d-467c-8c49-8365b51b1f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:00.480660 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdff41-b94d-467c-8c49-8365b51b1f11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:00.481839 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bdff41-b94d-467c-8c49-8365b51b1f11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:00.484765 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bdff41-b94d-467c-8c49-8365b51b1f11 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:00.502971 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1000/1979] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:16:00.504758 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdff41-b94d-467c-8c49-8365b51b1f11 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:00.509554 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d3bdff41-b94d-467c-8c49-8365b51b1f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:00.523452 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 980/1980] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:16:00 2022] POST /identity/v3/projects => generated 385 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:16:00.523452 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5c456-8006-4f1e-afb8-035c4a9b107e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:00.631876 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdff41-b94d-467c-8c49-8365b51b1f11 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:00.646610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5c456-8006-4f1e-afb8-035c4a9b107e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:00.654963 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f314a-211d-4b01-a733-f038c8548f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:00.656135 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f314a-211d-4b01-a733-f038c8548f40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:00.656630 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f314a-211d-4b01-a733-f038c8548f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:00.657102 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f314a-211d-4b01-a733-f038c8548f40 admin admin] PATH_INFO: `/v3/projects/bfbeeb5fb93c4b3e89fa20c8942115ef/users/7e4a016ffe4a493bb238e3c37c0ed541/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:00.686734 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f314a-211d-4b01-a733-f038c8548f40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfbeeb5fb93c4b3e89fa20c8942115ef, user_id=7e4a016ffe4a493bb238e3c37c0ed541, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:00.692145 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f314a-211d-4b01-a733-f038c8548f40 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:00.695581 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a4096-01d1-4d4e-a411-6ff3d55d6bfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:00.698094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a4096-01d1-4d4e-a411-6ff3d55d6bfe service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:00.698852 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a4096-01d1-4d4e-a411-6ff3d55d6bfe service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:00.699131 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a4096-01d1-4d4e-a411-6ff3d55d6bfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:00.730177 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:00.775125 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a4096-01d1-4d4e-a411-6ff3d55d6bfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:00.776995 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a4096-01d1-4d4e-a411-6ff3d55d6bfe service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:00.786795 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1001/1981] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:16:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:16:00.787647 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a4096-01d1-4d4e-a411-6ff3d55d6bfe service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:00.801461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 981/1982] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:16:00 2022] PUT /identity/v3/projects/bfbeeb5fb93c4b3e89fa20c8942115ef/users/7e4a016ffe4a493bb238e3c37c0ed541/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:16:00.803036 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a4096-01d1-4d4e-a411-6ff3d55d6bfe service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:00.804226 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f314a-211d-4b01-a733-f038c8548f40 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:00.906814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57990c4-6839-4ff2-8e1b-51ad0a6489e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:00.907861 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57990c4-6839-4ff2-8e1b-51ad0a6489e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:00.908358 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57990c4-6839-4ff2-8e1b-51ad0a6489e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:00.908844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57990c4-6839-4ff2-8e1b-51ad0a6489e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:00.909945 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57990c4-6839-4ff2-8e1b-51ad0a6489e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:00.910400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f314a-211d-4b01-a733-f038c8548f40 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:00.913600 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57990c4-6839-4ff2-8e1b-51ad0a6489e5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:00.923460 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287383f-425e-4ad4-821f-c2a7318f1945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:00.924126 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287383f-425e-4ad4-821f-c2a7318f1945 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:00.924417 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287383f-425e-4ad4-821f-c2a7318f1945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:00.924692 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287383f-425e-4ad4-821f-c2a7318f1945 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:00.925366 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287383f-425e-4ad4-821f-c2a7318f1945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:00.927111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287383f-425e-4ad4-821f-c2a7318f1945 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:00.939836 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 982/1983] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:16:00.944159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287383f-425e-4ad4-821f-c2a7318f1945 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:00.960931 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287383f-425e-4ad4-821f-c2a7318f1945 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:00.969304 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12190df9-758d-4189-88e5-88126b63979b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:00.970419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12190df9-758d-4189-88e5-88126b63979b admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:00.977210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12190df9-758d-4189-88e5-88126b63979b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:00.977210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12190df9-758d-4189-88e5-88126b63979b admin admin] PATH_INFO: `/v3/projects/bfbeeb5fb93c4b3e89fa20c8942115ef/users/7e4a016ffe4a493bb238e3c37c0ed541/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:00.977210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12190df9-758d-4189-88e5-88126b63979b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfbeeb5fb93c4b3e89fa20c8942115ef, user_id=7e4a016ffe4a493bb238e3c37c0ed541, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:00.980463 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12190df9-758d-4189-88e5-88126b63979b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:00.997999 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 983/1984] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:16:00 2022] PUT /identity/v3/projects/bfbeeb5fb93c4b3e89fa20c8942115ef/users/7e4a016ffe4a493bb238e3c37c0ed541/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:16:01.008439 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1002/1985] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:16:00 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:16:01.014154 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa4c76-3ca9-4904-aa06-6b7b5303c5d5 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:01.014154 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa4c76-3ca9-4904-aa06-6b7b5303c5d5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:01.014154 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa4c76-3ca9-4904-aa06-6b7b5303c5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:01.016590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57990c4-6839-4ff2-8e1b-51ad0a6489e5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:01.034476 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57990c4-6839-4ff2-8e1b-51ad0a6489e5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:01.036048 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:01.049189 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e8fa4c76-3ca9-4904-aa06-6b7b5303c5d5 None None] MFA Rules not processed for user `7e4a016ffe4a493bb238e3c37c0ed541`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:01.052091 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e8fa4c76-3ca9-4904-aa06-6b7b5303c5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:01.109992 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb02c0-3ae4-4af6-a61a-e508551c236c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:01.110646 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb02c0-3ae4-4af6-a61a-e508551c236c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:01.110920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb02c0-3ae4-4af6-a61a-e508551c236c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:01.111186 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb02c0-3ae4-4af6-a61a-e508551c236c admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:01.111828 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb02c0-3ae4-4af6-a61a-e508551c236c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:01.113510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb02c0-3ae4-4af6-a61a-e508551c236c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:01.125673 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1003/1986] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:16:01.130009 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb02c0-3ae4-4af6-a61a-e508551c236c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:01.142222 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb02c0-3ae4-4af6-a61a-e508551c236c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:01.147855 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a57855-7bd4-41dd-bdc1-7473466810ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:01.148473 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a57855-7bd4-41dd-bdc1-7473466810ac admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:01.148748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a57855-7bd4-41dd-bdc1-7473466810ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:01.149425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a57855-7bd4-41dd-bdc1-7473466810ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:01.150301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a57855-7bd4-41dd-bdc1-7473466810ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:01.152003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a57855-7bd4-41dd-bdc1-7473466810ac admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:01.165784 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1004/1987] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:16:01.166048 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 984/1988] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:01 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:16:01.170486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a57855-7bd4-41dd-bdc1-7473466810ac admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:01.182338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa4c76-3ca9-4904-aa06-6b7b5303c5d5 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:01.182632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a57855-7bd4-41dd-bdc1-7473466810ac admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:01.184097 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e8fa4c76-3ca9-4904-aa06-6b7b5303c5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:01.187637 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ecbb6-8273-4adc-8ff0-41a961673031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:01.188338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ecbb6-8273-4adc-8ff0-41a961673031 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:01.188610 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ecbb6-8273-4adc-8ff0-41a961673031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:01.188875 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ecbb6-8273-4adc-8ff0-41a961673031 admin admin] PATH_INFO: `/v3/projects/89fcec54c0884d8ba44caf78b72daaef/users/c8fc720a0b60464bb8e6456aa4e61b2a/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:01.208040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ecbb6-8273-4adc-8ff0-41a961673031 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89fcec54c0884d8ba44caf78b72daaef, user_id=c8fc720a0b60464bb8e6456aa4e61b2a, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:01.211983 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ecbb6-8273-4adc-8ff0-41a961673031 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:01.259316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa4c76-3ca9-4904-aa06-6b7b5303c5d5 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:01.265223 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa303c-16dc-447b-a8dd-db5ed591bf03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:01.265925 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa303c-16dc-447b-a8dd-db5ed591bf03 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:01.266198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa303c-16dc-447b-a8dd-db5ed591bf03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:01.266540 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa303c-16dc-447b-a8dd-db5ed591bf03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:01.277514 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aa303c-16dc-447b-a8dd-db5ed591bf03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:01.279038 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aa303c-16dc-447b-a8dd-db5ed591bf03 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:01.296194 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 985/1989] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:16:01 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:16:01.296439 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1005/1990] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:16:01 2022] PUT /identity/v3/projects/89fcec54c0884d8ba44caf78b72daaef/users/c8fc720a0b60464bb8e6456aa4e61b2a/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:16:01.307631 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8b0ad-08bc-4550-9d70-4ca106e4912e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:01.307945 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8b0ad-08bc-4550-9d70-4ca106e4912e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:01.308226 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8b0ad-08bc-4550-9d70-4ca106e4912e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:01.342327 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-28f8b0ad-08bc-4550-9d70-4ca106e4912e None None] MFA Rules not processed for user `c8fc720a0b60464bb8e6456aa4e61b2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:01.343738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-28f8b0ad-08bc-4550-9d70-4ca106e4912e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:01.433439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 986/1991] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:16:01.446442 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ecbb6-8273-4adc-8ff0-41a961673031 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:01.449523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8b0ad-08bc-4550-9d70-4ca106e4912e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:01.451430 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-28f8b0ad-08bc-4550-9d70-4ca106e4912e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:01.493515 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:01.499325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:01.557375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8b0ad-08bc-4550-9d70-4ca106e4912e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:01.578483 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769e4c5-75ee-4970-9223-57a4af8cd923 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:01.578483 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769e4c5-75ee-4970-9223-57a4af8cd923 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:01.578483 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769e4c5-75ee-4970-9223-57a4af8cd923 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:01.578483 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769e4c5-75ee-4970-9223-57a4af8cd923 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:01.592613 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769e4c5-75ee-4970-9223-57a4af8cd923 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:01.595413 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769e4c5-75ee-4970-9223-57a4af8cd923 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:01.608099 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ecbb6-8273-4adc-8ff0-41a961673031 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:01.618786 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02a6bb-86b0-4ba6-a592-4a8285f83ee5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:01.619441 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02a6bb-86b0-4ba6-a592-4a8285f83ee5 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:01.619938 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02a6bb-86b0-4ba6-a592-4a8285f83ee5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:01.620532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 987/1992] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:16:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:16:01.621149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02a6bb-86b0-4ba6-a592-4a8285f83ee5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:01.752514 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed02a6bb-86b0-4ba6-a592-4a8285f83ee5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:01.758260 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed02a6bb-86b0-4ba6-a592-4a8285f83ee5 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:01.770969 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1006/1993] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:16:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:16:11.623724 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=21 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:11.763538 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=40 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:14.802762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769e4c5-75ee-4970-9223-57a4af8cd923 service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:14.890974 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769e4c5-75ee-4970-9223-57a4af8cd923 service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:14.907973 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a9408-7935-4b0c-bd2c-905fc0837c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:14.907973 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a9408-7935-4b0c-bd2c-905fc0837c5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:14.907973 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a9408-7935-4b0c-bd2c-905fc0837c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:14.907973 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a9408-7935-4b0c-bd2c-905fc0837c5c admin admin] PATH_INFO: `/v3/users/567aa4145ac844669b802bb06faf7abc` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:14.909420 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778a9408-7935-4b0c-bd2c-905fc0837c5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=567aa4145ac844669b802bb06faf7abc)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:14.910163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778a9408-7935-4b0c-bd2c-905fc0837c5c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:15.142303 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-778a9408-7935-4b0c-bd2c-905fc0837c5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '567aa4145ac844669b802bb06faf7abc'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:15.188388 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-778a9408-7935-4b0c-bd2c-905fc0837c5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '567aa4145ac844669b802bb06faf7abc'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:15.206571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-778a9408-7935-4b0c-bd2c-905fc0837c5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '567aa4145ac844669b802bb06faf7abc'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:15.206571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-778a9408-7935-4b0c-bd2c-905fc0837c5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '567aa4145ac844669b802bb06faf7abc'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:15.208921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-778a9408-7935-4b0c-bd2c-905fc0837c5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '567aa4145ac844669b802bb06faf7abc'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:15.223956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 988/1994] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:16:14 2022] DELETE /identity/v3/users/567aa4145ac844669b802bb06faf7abc => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:16:15.229271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21acc5-c3cc-481a-b8ef-f43aa23df5a0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:15.229966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21acc5-c3cc-481a-b8ef-f43aa23df5a0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:15.229966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21acc5-c3cc-481a-b8ef-f43aa23df5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:15.250116 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6c21acc5-c3cc-481a-b8ef-f43aa23df5a0 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:15.251624 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6c21acc5-c3cc-481a-b8ef-f43aa23df5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:15.322083 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1007/1995] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:16:15.332024 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a9408-7935-4b0c-bd2c-905fc0837c5c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:15.333566 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-778a9408-7935-4b0c-bd2c-905fc0837c5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:15.426615 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a9408-7935-4b0c-bd2c-905fc0837c5c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:15.430634 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce9e4f5-c63e-4395-810c-204aa2f9f8ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:15.431727 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9e4f5-c63e-4395-810c-204aa2f9f8ba service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:15.431996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9e4f5-c63e-4395-810c-204aa2f9f8ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:15.432333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9e4f5-c63e-4395-810c-204aa2f9f8ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:15.442904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce9e4f5-c63e-4395-810c-204aa2f9f8ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:15.444363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce9e4f5-c63e-4395-810c-204aa2f9f8ba service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:15.455791 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 989/1996] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:16:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:16:15.942381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21acc5-c3cc-481a-b8ef-f43aa23df5a0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:15.943968 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6c21acc5-c3cc-481a-b8ef-f43aa23df5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:15.959598 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21acc5-c3cc-481a-b8ef-f43aa23df5a0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:15.973625 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05101894-95d2-4214-834a-098250b125e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:15.973625 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05101894-95d2-4214-834a-098250b125e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:15.973625 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05101894-95d2-4214-834a-098250b125e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:15.973625 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05101894-95d2-4214-834a-098250b125e8 admin admin] PATH_INFO: `/v3/projects/d78379f2ae4b4aad8c69deed4fdfb65f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:15.974051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05101894-95d2-4214-834a-098250b125e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d78379f2ae4b4aad8c69deed4fdfb65f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:15.974051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05101894-95d2-4214-834a-098250b125e8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:16.111545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-05101894-95d2-4214-834a-098250b125e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd78379f2ae4b4aad8c69deed4fdfb65f', 'request_id': 'req-05101894-95d2-4214-834a-098250b125e8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:16.124212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-05101894-95d2-4214-834a-098250b125e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd78379f2ae4b4aad8c69deed4fdfb65f', 'request_id': 'req-05101894-95d2-4214-834a-098250b125e8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:16.205246 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-05101894-95d2-4214-834a-098250b125e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd78379f2ae4b4aad8c69deed4fdfb65f', 'request_id': 'req-05101894-95d2-4214-834a-098250b125e8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:16.227775 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-05101894-95d2-4214-834a-098250b125e8 admin admin] The token cache is being invalidate because project d78379f2ae4b4aad8c69deed4fdfb65f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:16:16.229165 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-05101894-95d2-4214-834a-098250b125e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:16.231370 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-05101894-95d2-4214-834a-098250b125e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:16.235594 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1008/1997] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:16:15 2022] DELETE /identity/v3/projects/d78379f2ae4b4aad8c69deed4fdfb65f => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:16:16.240555 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce9e4f5-c63e-4395-810c-204aa2f9f8ba service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:16.255737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce9e4f5-c63e-4395-810c-204aa2f9f8ba service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:16.283371 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:16.331495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ec8b2-9fb6-4082-9c40-749c01e9b0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:16.332575 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ec8b2-9fb6-4082-9c40-749c01e9b0af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:16.333003 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ec8b2-9fb6-4082-9c40-749c01e9b0af admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:16.333358 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ec8b2-9fb6-4082-9c40-749c01e9b0af admin admin] PATH_INFO: `/v3/users/7e4a016ffe4a493bb238e3c37c0ed541` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:16.335387 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ec8b2-9fb6-4082-9c40-749c01e9b0af admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e4a016ffe4a493bb238e3c37c0ed541)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:16.339103 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ec8b2-9fb6-4082-9c40-749c01e9b0af admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:16.552577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-929ec8b2-9fb6-4082-9c40-749c01e9b0af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e4a016ffe4a493bb238e3c37c0ed541'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:16.625206 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-929ec8b2-9fb6-4082-9c40-749c01e9b0af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e4a016ffe4a493bb238e3c37c0ed541'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:16.662485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-929ec8b2-9fb6-4082-9c40-749c01e9b0af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e4a016ffe4a493bb238e3c37c0ed541'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:16.665450 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-929ec8b2-9fb6-4082-9c40-749c01e9b0af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e4a016ffe4a493bb238e3c37c0ed541'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:16.666737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-929ec8b2-9fb6-4082-9c40-749c01e9b0af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e4a016ffe4a493bb238e3c37c0ed541'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:16.682091 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 990/1998] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:16:16 2022] DELETE /identity/v3/users/7e4a016ffe4a493bb238e3c37c0ed541 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:16:16.750497 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05101894-95d2-4214-834a-098250b125e8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:16.752408 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-05101894-95d2-4214-834a-098250b125e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:16.769501 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05101894-95d2-4214-834a-098250b125e8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:16.852055 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066c402-59bd-4749-a9a1-17a172769df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:16.852950 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066c402-59bd-4749-a9a1-17a172769df6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:16.853711 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066c402-59bd-4749-a9a1-17a172769df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:16.855023 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066c402-59bd-4749-a9a1-17a172769df6 admin admin] PATH_INFO: `/v3/users/c8fc720a0b60464bb8e6456aa4e61b2a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:16.857265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066c402-59bd-4749-a9a1-17a172769df6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8fc720a0b60464bb8e6456aa4e61b2a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:16.859752 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066c402-59bd-4749-a9a1-17a172769df6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:16.964628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:17.071554 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3066c402-59bd-4749-a9a1-17a172769df6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8fc720a0b60464bb8e6456aa4e61b2a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:17.166317 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3066c402-59bd-4749-a9a1-17a172769df6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8fc720a0b60464bb8e6456aa4e61b2a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:17.178306 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3066c402-59bd-4749-a9a1-17a172769df6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8fc720a0b60464bb8e6456aa4e61b2a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:17.195320 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3066c402-59bd-4749-a9a1-17a172769df6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8fc720a0b60464bb8e6456aa4e61b2a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:17.197153 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3066c402-59bd-4749-a9a1-17a172769df6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8fc720a0b60464bb8e6456aa4e61b2a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:17.206815 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1009/1999] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:16:16 2022] DELETE /identity/v3/users/c8fc720a0b60464bb8e6456aa4e61b2a => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:16:17.211391 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a232c-95ec-4748-bc31-f3babf2363a6 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:17.211713 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a232c-95ec-4748-bc31-f3babf2363a6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:17.212031 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a232c-95ec-4748-bc31-f3babf2363a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:17.231957 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8b1a232c-95ec-4748-bc31-f3babf2363a6 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:17.233188 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8b1a232c-95ec-4748-bc31-f3babf2363a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:17.296441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 991/2000] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:16:17.307007 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066c402-59bd-4749-a9a1-17a172769df6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:17.311670 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3066c402-59bd-4749-a9a1-17a172769df6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:17.355980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a232c-95ec-4748-bc31-f3babf2363a6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:17.359200 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8b1a232c-95ec-4748-bc31-f3babf2363a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:17.375609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066c402-59bd-4749-a9a1-17a172769df6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:17.379468 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9a189-8f3f-4fbd-a7d9-7d4c54427c6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:17.380145 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9a189-8f3f-4fbd-a7d9-7d4c54427c6f service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:17.380365 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9a189-8f3f-4fbd-a7d9-7d4c54427c6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:17.380835 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9a189-8f3f-4fbd-a7d9-7d4c54427c6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:17.385779 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a232c-95ec-4748-bc31-f3babf2363a6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:17.393357 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a1006-bf4a-405b-bf67-23f4f4fc2fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:17.395166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a1006-bf4a-405b-bf67-23f4f4fc2fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:17.395544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b9a189-8f3f-4fbd-a7d9-7d4c54427c6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:17.395834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a1006-bf4a-405b-bf67-23f4f4fc2fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:17.396322 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a1006-bf4a-405b-bf67-23f4f4fc2fa5 admin admin] PATH_INFO: `/v3/projects/bfbeeb5fb93c4b3e89fa20c8942115ef` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:17.397284 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b9a189-8f3f-4fbd-a7d9-7d4c54427c6f service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:17.398288 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a1006-bf4a-405b-bf67-23f4f4fc2fa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfbeeb5fb93c4b3e89fa20c8942115ef)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:17.403379 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a1006-bf4a-405b-bf67-23f4f4fc2fa5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:17.415434 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1010/2001] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:16:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:16:17.549797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:17.569005 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-112a1006-bf4a-405b-bf67-23f4f4fc2fa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfbeeb5fb93c4b3e89fa20c8942115ef', 'request_id': 'req-112a1006-bf4a-405b-bf67-23f4f4fc2fa5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:17.680155 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-112a1006-bf4a-405b-bf67-23f4f4fc2fa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfbeeb5fb93c4b3e89fa20c8942115ef', 'request_id': 'req-112a1006-bf4a-405b-bf67-23f4f4fc2fa5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:17.688416 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-112a1006-bf4a-405b-bf67-23f4f4fc2fa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfbeeb5fb93c4b3e89fa20c8942115ef', 'request_id': 'req-112a1006-bf4a-405b-bf67-23f4f4fc2fa5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:17.700196 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-112a1006-bf4a-405b-bf67-23f4f4fc2fa5 admin admin] The token cache is being invalidate because project bfbeeb5fb93c4b3e89fa20c8942115ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:16:17.700490 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-112a1006-bf4a-405b-bf67-23f4f4fc2fa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:17.701356 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-112a1006-bf4a-405b-bf67-23f4f4fc2fa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:17.703216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 992/2002] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:16:17 2022] DELETE /identity/v3/projects/bfbeeb5fb93c4b3e89fa20c8942115ef => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:16:17.709238 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6911b-77b6-4808-8d5f-eaba814f3a85 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:17.712900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6911b-77b6-4808-8d5f-eaba814f3a85 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:17.712900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6911b-77b6-4808-8d5f-eaba814f3a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:17.746322 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8aa6911b-77b6-4808-8d5f-eaba814f3a85 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:17.750557 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8aa6911b-77b6-4808-8d5f-eaba814f3a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:17.878387 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1011/2003] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:16:17.889201 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c9869-8df5-4d5f-ba39-4201d85562bb None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:17.889883 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c9869-8df5-4d5f-ba39-4201d85562bb None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:17.890386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c9869-8df5-4d5f-ba39-4201d85562bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:17.934076 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b06c9869-8df5-4d5f-ba39-4201d85562bb None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:17.936414 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b06c9869-8df5-4d5f-ba39-4201d85562bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:17.952326 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 993/2004] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:16:17.959210 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6911b-77b6-4808-8d5f-eaba814f3a85 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:17.978237 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6911b-77b6-4808-8d5f-eaba814f3a85 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:17.986784 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f66e6a-fdae-4f84-abc5-137b8e14ddec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:17.987464 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f66e6a-fdae-4f84-abc5-137b8e14ddec admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:17.987937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f66e6a-fdae-4f84-abc5-137b8e14ddec admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:17.988875 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f66e6a-fdae-4f84-abc5-137b8e14ddec admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:17.990524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f66e6a-fdae-4f84-abc5-137b8e14ddec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:16:17.991121 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f66e6a-fdae-4f84-abc5-137b8e14ddec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:17.991610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c9869-8df5-4d5f-ba39-4201d85562bb None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:17.995204 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f66e6a-fdae-4f84-abc5-137b8e14ddec admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:17.995353 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b06c9869-8df5-4d5f-ba39-4201d85562bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:18.025656 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c9869-8df5-4d5f-ba39-4201d85562bb None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:18.026093 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1012/2005] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:16:18.041685 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f66e6a-fdae-4f84-abc5-137b8e14ddec admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:18.042000 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fe4ce-f1e5-4f0d-b602-80d5e747b36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:18.042000 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fe4ce-f1e5-4f0d-b602-80d5e747b36e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:18.042344 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fe4ce-f1e5-4f0d-b602-80d5e747b36e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:18.042344 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fe4ce-f1e5-4f0d-b602-80d5e747b36e admin admin] PATH_INFO: `/v3/projects/89fcec54c0884d8ba44caf78b72daaef` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:18.044049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32fe4ce-f1e5-4f0d-b602-80d5e747b36e admin admin] RBAC: Authorizing `identity:delete_project(project_id=89fcec54c0884d8ba44caf78b72daaef)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:18.045879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32fe4ce-f1e5-4f0d-b602-80d5e747b36e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:18.054170 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f66e6a-fdae-4f84-abc5-137b8e14ddec admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:18.067090 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5c128-5dca-4847-96aa-a8bf7cce8ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:18.068246 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5c128-5dca-4847-96aa-a8bf7cce8ac1 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:18.068852 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5c128-5dca-4847-96aa-a8bf7cce8ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:18.069386 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5c128-5dca-4847-96aa-a8bf7cce8ac1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:18.071242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5c128-5dca-4847-96aa-a8bf7cce8ac1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:18.074921 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5c128-5dca-4847-96aa-a8bf7cce8ac1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:18.180298 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e32fe4ce-f1e5-4f0d-b602-80d5e747b36e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89fcec54c0884d8ba44caf78b72daaef', 'request_id': 'req-e32fe4ce-f1e5-4f0d-b602-80d5e747b36e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:18.195852 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1013/2006] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:16:18 2022] POST /identity/v3/projects => generated 371 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:16:18.200286 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5c128-5dca-4847-96aa-a8bf7cce8ac1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:18.213623 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:18.276475 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e32fe4ce-f1e5-4f0d-b602-80d5e747b36e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89fcec54c0884d8ba44caf78b72daaef', 'request_id': 'req-e32fe4ce-f1e5-4f0d-b602-80d5e747b36e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:18.276790 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5c128-5dca-4847-96aa-a8bf7cce8ac1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:18.282536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af98ce6d-b308-4c17-9d49-430164e4f929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:18.283345 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98ce6d-b308-4c17-9d49-430164e4f929 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:18.283700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e32fe4ce-f1e5-4f0d-b602-80d5e747b36e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89fcec54c0884d8ba44caf78b72daaef', 'request_id': 'req-e32fe4ce-f1e5-4f0d-b602-80d5e747b36e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:18.283900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98ce6d-b308-4c17-9d49-430164e4f929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:18.284171 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98ce6d-b308-4c17-9d49-430164e4f929 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:18.284989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af98ce6d-b308-4c17-9d49-430164e4f929 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:18.287578 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af98ce6d-b308-4c17-9d49-430164e4f929 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:18.298060 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e32fe4ce-f1e5-4f0d-b602-80d5e747b36e admin admin] The token cache is being invalidate because project 89fcec54c0884d8ba44caf78b72daaef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:16:18.301915 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e32fe4ce-f1e5-4f0d-b602-80d5e747b36e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:18.302780 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e32fe4ce-f1e5-4f0d-b602-80d5e747b36e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:18.304401 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 994/2007] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:16:17 2022] DELETE /identity/v3/projects/89fcec54c0884d8ba44caf78b72daaef => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:16:18.310688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff2277-92db-4384-b52b-cf1eab3c1953 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:18.310978 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff2277-92db-4384-b52b-cf1eab3c1953 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:18.311256 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff2277-92db-4384-b52b-cf1eab3c1953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:18.335109 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-50ff2277-92db-4384-b52b-cf1eab3c1953 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:18.337057 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-50ff2277-92db-4384-b52b-cf1eab3c1953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:18.359121 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 995/2008] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:16:18.361755 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f0b22-3c41-4af4-a775-7af048d73af2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:18.362188 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f0b22-3c41-4af4-a775-7af048d73af2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:18.362626 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f0b22-3c41-4af4-a775-7af048d73af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:18.383907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-206f0b22-3c41-4af4-a775-7af048d73af2 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:18.385141 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-206f0b22-3c41-4af4-a775-7af048d73af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:18.395700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 996/2009] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:16:18.399947 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f0b22-3c41-4af4-a775-7af048d73af2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:18.404437 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1014/2010] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:16:18 2022] POST /identity/v3/users => generated 286 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:16:18.405502 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af98ce6d-b308-4c17-9d49-430164e4f929 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:18.407205 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-af98ce6d-b308-4c17-9d49-430164e4f929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:18.410307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f0b22-3c41-4af4-a775-7af048d73af2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:18.415336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad2454-ff0a-4117-a04c-8bf1cebf9f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:18.415956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad2454-ff0a-4117-a04c-8bf1cebf9f96 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:18.416230 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad2454-ff0a-4117-a04c-8bf1cebf9f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:18.416499 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad2454-ff0a-4117-a04c-8bf1cebf9f96 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:18.417180 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad2454-ff0a-4117-a04c-8bf1cebf9f96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:16:18.417486 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad2454-ff0a-4117-a04c-8bf1cebf9f96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:18.419229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad2454-ff0a-4117-a04c-8bf1cebf9f96 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:18.421391 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af98ce6d-b308-4c17-9d49-430164e4f929 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:18.426560 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164de2e-035e-4c55-8f81-b9bd6e6fe7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:18.427260 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164de2e-035e-4c55-8f81-b9bd6e6fe7a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:18.428257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164de2e-035e-4c55-8f81-b9bd6e6fe7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:18.428257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164de2e-035e-4c55-8f81-b9bd6e6fe7a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:18.428865 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c164de2e-035e-4c55-8f81-b9bd6e6fe7a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:18.431996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c164de2e-035e-4c55-8f81-b9bd6e6fe7a7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:18.433572 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 997/2011] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 17:16:18.437876 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad2454-ff0a-4117-a04c-8bf1cebf9f96 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:18.452560 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1015/2012] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:16:18.452955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad2454-ff0a-4117-a04c-8bf1cebf9f96 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:18.453669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164de2e-035e-4c55-8f81-b9bd6e6fe7a7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:18.459149 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c4f11-5973-4940-b2f1-baf88676a50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:18.466572 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c4f11-5973-4940-b2f1-baf88676a50c admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:18.466572 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c4f11-5973-4940-b2f1-baf88676a50c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:18.466572 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c4f11-5973-4940-b2f1-baf88676a50c admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:18.466572 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c4f11-5973-4940-b2f1-baf88676a50c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:18.466572 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c4f11-5973-4940-b2f1-baf88676a50c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:18.467037 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164de2e-035e-4c55-8f81-b9bd6e6fe7a7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:18.471086 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d73d75-9764-4e0f-8fa3-bd44f1883a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:18.472182 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d73d75-9764-4e0f-8fa3-bd44f1883a92 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:18.472182 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d73d75-9764-4e0f-8fa3-bd44f1883a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:18.472380 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d73d75-9764-4e0f-8fa3-bd44f1883a92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:18.473101 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d73d75-9764-4e0f-8fa3-bd44f1883a92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:18.474613 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d73d75-9764-4e0f-8fa3-bd44f1883a92 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:18.486150 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1016/2013] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:16:18.499656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d73d75-9764-4e0f-8fa3-bd44f1883a92 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:18.513528 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d73d75-9764-4e0f-8fa3-bd44f1883a92 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:18.519495 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93daf0-7b15-46af-9e71-5020bf4ff114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:18.520359 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93daf0-7b15-46af-9e71-5020bf4ff114 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:18.520738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93daf0-7b15-46af-9e71-5020bf4ff114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:18.521101 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93daf0-7b15-46af-9e71-5020bf4ff114 admin admin] PATH_INFO: `/v3/projects/f827cc75b1c449f6a82d5c23af901fa0/users/44132cbd18f2470dbe9de6b5125d7db5/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:18.538533 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93daf0-7b15-46af-9e71-5020bf4ff114 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f827cc75b1c449f6a82d5c23af901fa0, user_id=44132cbd18f2470dbe9de6b5125d7db5, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:18.543714 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93daf0-7b15-46af-9e71-5020bf4ff114 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:18.593288 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 998/2014] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:16:18 2022] POST /identity/v3/projects => generated 357 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:16:18.597729 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c4f11-5973-4940-b2f1-baf88676a50c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:18.612184 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1017/2015] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:16:18 2022] PUT /identity/v3/projects/f827cc75b1c449f6a82d5c23af901fa0/users/44132cbd18f2470dbe9de6b5125d7db5/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:16:18.619084 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7bf6e8-c752-4a11-9b78-ebccaca98fc0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:18.620482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7bf6e8-c752-4a11-9b78-ebccaca98fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:18.622217 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7bf6e8-c752-4a11-9b78-ebccaca98fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:18.643825 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-fb7bf6e8-c752-4a11-9b78-ebccaca98fc0 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:18.645419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fb7bf6e8-c752-4a11-9b78-ebccaca98fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:18.721271 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c4f11-5973-4940-b2f1-baf88676a50c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:18.734403 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:18.734723 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1018/2016] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:16:18.754247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7bf6e8-c752-4a11-9b78-ebccaca98fc0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:18.756205 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fb7bf6e8-c752-4a11-9b78-ebccaca98fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:18.821520 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba7b37-917f-4938-aae7-726048406b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:18.823362 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba7b37-917f-4938-aae7-726048406b08 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:18.823931 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba7b37-917f-4938-aae7-726048406b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:18.824443 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba7b37-917f-4938-aae7-726048406b08 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:18.825613 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fba7b37-917f-4938-aae7-726048406b08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:18.829193 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fba7b37-917f-4938-aae7-726048406b08 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:18.829466 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7bf6e8-c752-4a11-9b78-ebccaca98fc0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:18.834672 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df92ca-c1e8-4fbc-92f6-e6db18221588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:18.835525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df92ca-c1e8-4fbc-92f6-e6db18221588 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:18.835949 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df92ca-c1e8-4fbc-92f6-e6db18221588 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:18.836447 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df92ca-c1e8-4fbc-92f6-e6db18221588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:18.856936 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df92ca-c1e8-4fbc-92f6-e6db18221588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:18.860544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df92ca-c1e8-4fbc-92f6-e6db18221588 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:18.887656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1019/2017] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:16:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:16:18.926010 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 999/2018] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:16:18 2022] POST /identity/v3/users => generated 279 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:16:18.932468 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df92ca-c1e8-4fbc-92f6-e6db18221588 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:18.945235 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df92ca-c1e8-4fbc-92f6-e6db18221588 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:18.953983 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af1eabc-5404-4501-8950-eaba4de9e4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:18.955098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1eabc-5404-4501-8950-eaba4de9e4b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:18.955558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1eabc-5404-4501-8950-eaba4de9e4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:18.956003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1eabc-5404-4501-8950-eaba4de9e4b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:18.957113 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af1eabc-5404-4501-8950-eaba4de9e4b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:18.960010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af1eabc-5404-4501-8950-eaba4de9e4b3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:18.973142 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1020/2019] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:16:18.983957 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba7b37-917f-4938-aae7-726048406b08 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:18.995116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba7b37-917f-4938-aae7-726048406b08 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:19.000146 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fbbfc5-3325-424f-b693-2633c41725fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:19.000747 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbbfc5-3325-424f-b693-2633c41725fe admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:19.001020 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbbfc5-3325-424f-b693-2633c41725fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:19.001287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbbfc5-3325-424f-b693-2633c41725fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:19.001928 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fbbfc5-3325-424f-b693-2633c41725fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:19.003558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fbbfc5-3325-424f-b693-2633c41725fe admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:19.015024 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1000/2020] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:16:19.019649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af1eabc-5404-4501-8950-eaba4de9e4b3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:19.031238 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af1eabc-5404-4501-8950-eaba4de9e4b3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:19.036142 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390b757-a333-4984-8205-e54095d292cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:19.036809 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390b757-a333-4984-8205-e54095d292cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:19.037091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390b757-a333-4984-8205-e54095d292cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:19.037366 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390b757-a333-4984-8205-e54095d292cb admin admin] PATH_INFO: `/v3/projects/b389ea9dd5ae4993b48424b0ac24c1af/users/7cdb7160092f4381890bb5a0cb2964ac/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:19.085960 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c390b757-a333-4984-8205-e54095d292cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b389ea9dd5ae4993b48424b0ac24c1af, user_id=7cdb7160092f4381890bb5a0cb2964ac, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:19.092724 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c390b757-a333-4984-8205-e54095d292cb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:19.235841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1021/2021] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:16:19 2022] PUT /identity/v3/projects/b389ea9dd5ae4993b48424b0ac24c1af/users/7cdb7160092f4381890bb5a0cb2964ac/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:16:19.254063 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59573448-5c50-4e09-baa9-70befb8ca251 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:19.254063 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59573448-5c50-4e09-baa9-70befb8ca251 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:19.254063 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59573448-5c50-4e09-baa9-70befb8ca251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:19.303799 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-59573448-5c50-4e09-baa9-70befb8ca251 None None] MFA Rules not processed for user `7cdb7160092f4381890bb5a0cb2964ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:19.306057 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-59573448-5c50-4e09-baa9-70befb8ca251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:19.319616 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:19.384325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1001/2022] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:19 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:16:19.389433 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390b757-a333-4984-8205-e54095d292cb admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:19.458304 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390b757-a333-4984-8205-e54095d292cb admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:19.463802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6889a-d548-4f3f-a3fa-7c75106e7265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:19.464443 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6889a-d548-4f3f-a3fa-7c75106e7265 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:19.464725 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6889a-d548-4f3f-a3fa-7c75106e7265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:19.464999 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6889a-d548-4f3f-a3fa-7c75106e7265 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:19.465652 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a6889a-d548-4f3f-a3fa-7c75106e7265 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:19.467733 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a6889a-d548-4f3f-a3fa-7c75106e7265 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:19.613720 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1022/2023] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:16:19 2022] POST /identity/v3/projects => generated 357 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:16:19.620401 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59573448-5c50-4e09-baa9-70befb8ca251 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:19.635778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59573448-5c50-4e09-baa9-70befb8ca251 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:19.700922 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64547688-de05-4d00-8703-0cdd7fc91b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:19.701537 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64547688-de05-4d00-8703-0cdd7fc91b8b admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:19.701816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64547688-de05-4d00-8703-0cdd7fc91b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:19.702089 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64547688-de05-4d00-8703-0cdd7fc91b8b admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:19.702804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64547688-de05-4d00-8703-0cdd7fc91b8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:19.704751 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64547688-de05-4d00-8703-0cdd7fc91b8b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:19.846397 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1002/2024] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:16:19 2022] POST /identity/v3/users => generated 285 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:16:19.849332 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6889a-d548-4f3f-a3fa-7c75106e7265 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:19.851815 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:19.862725 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6889a-d548-4f3f-a3fa-7c75106e7265 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:19.870959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb212c0-03c9-4339-8121-447f8f622bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:19.871807 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb212c0-03c9-4339-8121-447f8f622bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:19.876935 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb212c0-03c9-4339-8121-447f8f622bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:19.876935 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb212c0-03c9-4339-8121-447f8f622bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:19.876935 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb212c0-03c9-4339-8121-447f8f622bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:19.876935 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb212c0-03c9-4339-8121-447f8f622bd8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:19.888794 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1023/2025] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:16:19.893256 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64547688-de05-4d00-8703-0cdd7fc91b8b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:19.905923 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64547688-de05-4d00-8703-0cdd7fc91b8b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:19.911115 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f1572-f607-40ac-91b1-f0b07c901850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:19.911890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f1572-f607-40ac-91b1-f0b07c901850 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:19.912233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f1572-f607-40ac-91b1-f0b07c901850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:19.912582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f1572-f607-40ac-91b1-f0b07c901850 admin admin] PATH_INFO: `/v3/projects/f5017fe9f5c44a8cbc9e2a54ca42fd95/users/75a3e1dfb29747c28f3feb2a48329c6b/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:19.930914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f1572-f607-40ac-91b1-f0b07c901850 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5017fe9f5c44a8cbc9e2a54ca42fd95, user_id=75a3e1dfb29747c28f3feb2a48329c6b, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:19.934694 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f1572-f607-40ac-91b1-f0b07c901850 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:20.018605 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1003/2026] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:16:19 2022] PUT /identity/v3/projects/f5017fe9f5c44a8cbc9e2a54ca42fd95/users/75a3e1dfb29747c28f3feb2a48329c6b/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:16:20.026305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb212c0-03c9-4339-8121-447f8f622bd8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:20.044716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb212c0-03c9-4339-8121-447f8f622bd8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:20.138837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484c43da-fdb2-427c-a54a-6b090011e0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:20.139542 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c43da-fdb2-427c-a54a-6b090011e0ba admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:20.139846 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c43da-fdb2-427c-a54a-6b090011e0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:20.140745 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c43da-fdb2-427c-a54a-6b090011e0ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:20.141726 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484c43da-fdb2-427c-a54a-6b090011e0ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:20.144069 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484c43da-fdb2-427c-a54a-6b090011e0ba admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:20.156693 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1024/2027] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:16:20.161651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f1572-f607-40ac-91b1-f0b07c901850 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:20.173719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f1572-f607-40ac-91b1-f0b07c901850 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:20.180582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f87b0-1d1b-4248-9dc7-bda7025895f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:20.180582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f87b0-1d1b-4248-9dc7-bda7025895f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:20.180582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f87b0-1d1b-4248-9dc7-bda7025895f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:20.180582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f87b0-1d1b-4248-9dc7-bda7025895f1 admin admin] PATH_INFO: `/v3/projects/f5017fe9f5c44a8cbc9e2a54ca42fd95/users/75a3e1dfb29747c28f3feb2a48329c6b/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:20.182656 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f87b0-1d1b-4248-9dc7-bda7025895f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5017fe9f5c44a8cbc9e2a54ca42fd95, user_id=75a3e1dfb29747c28f3feb2a48329c6b, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:20.185799 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f87b0-1d1b-4248-9dc7-bda7025895f1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:20.196474 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1004/2028] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:16:20 2022] PUT /identity/v3/projects/f5017fe9f5c44a8cbc9e2a54ca42fd95/users/75a3e1dfb29747c28f3feb2a48329c6b/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:16:20.207416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2625b72-9084-4f43-ad05-20e1cc96270b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:20.207867 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2625b72-9084-4f43-ad05-20e1cc96270b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:20.210814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2625b72-9084-4f43-ad05-20e1cc96270b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:20.248649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b2625b72-9084-4f43-ad05-20e1cc96270b None None] MFA Rules not processed for user `75a3e1dfb29747c28f3feb2a48329c6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:20.250163 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b2625b72-9084-4f43-ad05-20e1cc96270b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:20.283416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:20.361871 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1025/2029] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:20 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:16:20.371425 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f87b0-1d1b-4248-9dc7-bda7025895f1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:20.373141 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d88f87b0-1d1b-4248-9dc7-bda7025895f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:20.442664 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3ada0-9efa-403f-b980-4dc37f5b0006 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:20.443125 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3ada0-9efa-403f-b980-4dc37f5b0006 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:20.443533 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3ada0-9efa-403f-b980-4dc37f5b0006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:20.450493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f87b0-1d1b-4248-9dc7-bda7025895f1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:20.454566 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41cb1a-858d-4f60-9131-03a1adc5ddbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:20.455192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41cb1a-858d-4f60-9131-03a1adc5ddbe service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:20.455468 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41cb1a-858d-4f60-9131-03a1adc5ddbe service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:20.455730 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41cb1a-858d-4f60-9131-03a1adc5ddbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:20.467320 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41cb1a-858d-4f60-9131-03a1adc5ddbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:20.468967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41cb1a-858d-4f60-9131-03a1adc5ddbe service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:20.480851 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1005/2030] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:16:20 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:16:20.490332 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-34a3ada0-9efa-403f-b980-4dc37f5b0006 None None] MFA Rules not processed for user `44132cbd18f2470dbe9de6b5125d7db5`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:20.495041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-34a3ada0-9efa-403f-b980-4dc37f5b0006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:20.575191 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1026/2031] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:16:20.602690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41cb1a-858d-4f60-9131-03a1adc5ddbe service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:20.618812 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41cb1a-858d-4f60-9131-03a1adc5ddbe service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:20.625420 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9364e471-1b53-4588-be6b-cfce37c60f1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:20.626502 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364e471-1b53-4588-be6b-cfce37c60f1f service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:20.626999 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364e471-1b53-4588-be6b-cfce37c60f1f service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:20.634607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364e471-1b53-4588-be6b-cfce37c60f1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:20.639822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9364e471-1b53-4588-be6b-cfce37c60f1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:20.641383 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9364e471-1b53-4588-be6b-cfce37c60f1f service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:20.652564 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1006/2032] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:16:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:16:20.672205 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3ada0-9efa-403f-b980-4dc37f5b0006 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:20.673808 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-34a3ada0-9efa-403f-b980-4dc37f5b0006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:20.693023 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3ada0-9efa-403f-b980-4dc37f5b0006 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:20.697277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbbda4-1a7c-4fb7-8622-3721e5649f89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:20.697935 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbbda4-1a7c-4fb7-8622-3721e5649f89 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:20.698212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbbda4-1a7c-4fb7-8622-3721e5649f89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:20.698704 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbbda4-1a7c-4fb7-8622-3721e5649f89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:20.784889 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cbbda4-1a7c-4fb7-8622-3721e5649f89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:20.786969 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cbbda4-1a7c-4fb7-8622-3721e5649f89 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:20.801556 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1027/2033] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:16:20 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:16:27.038571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300550e-73e1-4989-a8d9-4cfa64bad985 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:27.039601 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300550e-73e1-4989-a8d9-4cfa64bad985 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:27.039684 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300550e-73e1-4989-a8d9-4cfa64bad985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:27.059596 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f300550e-73e1-4989-a8d9-4cfa64bad985 None None] MFA Rules not processed for user `44132cbd18f2470dbe9de6b5125d7db5`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:27.060859 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f300550e-73e1-4989-a8d9-4cfa64bad985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:27.068428 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1007/2034] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:16:30.794799 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=79 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:31.145209 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbbda4-1a7c-4fb7-8622-3721e5649f89 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:31.146704 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-62cbbda4-1a7c-4fb7-8622-3721e5649f89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:31.283040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbbda4-1a7c-4fb7-8622-3721e5649f89 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:31.291229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3d675-9adb-4d3b-a0f0-58fb8e2f6370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:31.292954 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3d675-9adb-4d3b-a0f0-58fb8e2f6370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:31.292954 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3d675-9adb-4d3b-a0f0-58fb8e2f6370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:31.292954 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3d675-9adb-4d3b-a0f0-58fb8e2f6370 admin admin] PATH_INFO: `/v3/users/afbc55d3fcdc4782b7e0cafbf38de5b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:31.299907 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3d675-9adb-4d3b-a0f0-58fb8e2f6370 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afbc55d3fcdc4782b7e0cafbf38de5b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:31.305214 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3d675-9adb-4d3b-a0f0-58fb8e2f6370 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:31.589853 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4ea3d675-9adb-4d3b-a0f0-58fb8e2f6370 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afbc55d3fcdc4782b7e0cafbf38de5b6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:31.690137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4ea3d675-9adb-4d3b-a0f0-58fb8e2f6370 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afbc55d3fcdc4782b7e0cafbf38de5b6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:31.699976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4ea3d675-9adb-4d3b-a0f0-58fb8e2f6370 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afbc55d3fcdc4782b7e0cafbf38de5b6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:31.715652 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4ea3d675-9adb-4d3b-a0f0-58fb8e2f6370 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afbc55d3fcdc4782b7e0cafbf38de5b6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:31.716433 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4ea3d675-9adb-4d3b-a0f0-58fb8e2f6370 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afbc55d3fcdc4782b7e0cafbf38de5b6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:31.725363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1028/2035] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:16:31 2022] DELETE /identity/v3/users/afbc55d3fcdc4782b7e0cafbf38de5b6 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:16:31.730397 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac0bb5-8514-4478-80f0-c4796b0ef4d6 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:31.730947 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac0bb5-8514-4478-80f0-c4796b0ef4d6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:31.731188 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac0bb5-8514-4478-80f0-c4796b0ef4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:31.759601 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8bac0bb5-8514-4478-80f0-c4796b0ef4d6 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:31.762239 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8bac0bb5-8514-4478-80f0-c4796b0ef4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:31.830928 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1008/2036] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:16:31.841177 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3d675-9adb-4d3b-a0f0-58fb8e2f6370 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:31.843066 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4ea3d675-9adb-4d3b-a0f0-58fb8e2f6370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:31.903362 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3d675-9adb-4d3b-a0f0-58fb8e2f6370 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:31.907193 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b8fff-4262-4516-8c5d-b880c0504493 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:31.907977 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b8fff-4262-4516-8c5d-b880c0504493 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:31.908327 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b8fff-4262-4516-8c5d-b880c0504493 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:31.908704 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b8fff-4262-4516-8c5d-b880c0504493 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:31.923144 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b8fff-4262-4516-8c5d-b880c0504493 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:31.923144 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b8fff-4262-4516-8c5d-b880c0504493 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:31.934037 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1029/2037] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:16:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:16:32.577616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac0bb5-8514-4478-80f0-c4796b0ef4d6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:32.579399 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8bac0bb5-8514-4478-80f0-c4796b0ef4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:32.595864 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac0bb5-8514-4478-80f0-c4796b0ef4d6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:32.602410 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2068ac-185a-47e7-af1e-07f208be8265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:32.603432 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2068ac-185a-47e7-af1e-07f208be8265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:32.603776 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2068ac-185a-47e7-af1e-07f208be8265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:32.606830 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2068ac-185a-47e7-af1e-07f208be8265 admin admin] PATH_INFO: `/v3/projects/3940b616cbc448ab9d6eb31560814e90` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:32.610195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2068ac-185a-47e7-af1e-07f208be8265 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3940b616cbc448ab9d6eb31560814e90)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:32.612286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2068ac-185a-47e7-af1e-07f208be8265 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:32.794820 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:32.802080 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8d2068ac-185a-47e7-af1e-07f208be8265 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3940b616cbc448ab9d6eb31560814e90', 'request_id': 'req-8d2068ac-185a-47e7-af1e-07f208be8265'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:32.813829 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b8fff-4262-4516-8c5d-b880c0504493 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:32.815443 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e58b8fff-4262-4516-8c5d-b880c0504493 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:32.829003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b8fff-4262-4516-8c5d-b880c0504493 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:32.847692 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:32.868579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8d2068ac-185a-47e7-af1e-07f208be8265 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3940b616cbc448ab9d6eb31560814e90', 'request_id': 'req-8d2068ac-185a-47e7-af1e-07f208be8265'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:32.876748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8d2068ac-185a-47e7-af1e-07f208be8265 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3940b616cbc448ab9d6eb31560814e90', 'request_id': 'req-8d2068ac-185a-47e7-af1e-07f208be8265'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:32.892588 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8d2068ac-185a-47e7-af1e-07f208be8265 admin admin] The token cache is being invalidate because project 3940b616cbc448ab9d6eb31560814e90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:16:32.892940 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8d2068ac-185a-47e7-af1e-07f208be8265 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:32.902777 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8d2068ac-185a-47e7-af1e-07f208be8265 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:32.902777 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1009/2038] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:16:32 2022] DELETE /identity/v3/projects/3940b616cbc448ab9d6eb31560814e90 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:16:32.903781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2efedd-271c-405e-b0c8-a955c0773919 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:32.904069 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2efedd-271c-405e-b0c8-a955c0773919 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:32.904359 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2efedd-271c-405e-b0c8-a955c0773919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:32.909375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783dde2a-1221-4345-b0ff-c24ae1a165d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:32.910422 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dde2a-1221-4345-b0ff-c24ae1a165d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:32.911186 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dde2a-1221-4345-b0ff-c24ae1a165d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:32.911677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dde2a-1221-4345-b0ff-c24ae1a165d6 admin admin] PATH_INFO: `/v3/users/210be08699fb4653a32b6538aae2ebff` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:32.913337 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783dde2a-1221-4345-b0ff-c24ae1a165d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=210be08699fb4653a32b6538aae2ebff)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:32.916835 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783dde2a-1221-4345-b0ff-c24ae1a165d6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:32.936217 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8d2efedd-271c-405e-b0c8-a955c0773919 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:32.937574 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8d2efedd-271c-405e-b0c8-a955c0773919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:32.950603 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1010/2039] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:16:32.955412 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380ab99-51ab-440d-9ba5-0aac9263c501 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:32.955730 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380ab99-51ab-440d-9ba5-0aac9263c501 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:32.956000 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380ab99-51ab-440d-9ba5-0aac9263c501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:32.977144 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f380ab99-51ab-440d-9ba5-0aac9263c501 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:32.978422 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f380ab99-51ab-440d-9ba5-0aac9263c501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:32.990672 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1011/2040] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:16:32.992645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380ab99-51ab-440d-9ba5-0aac9263c501 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:33.006052 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380ab99-51ab-440d-9ba5-0aac9263c501 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:33.011098 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaafcf20-d854-4df6-b6b4-2131ec47083e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:33.011982 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafcf20-d854-4df6-b6b4-2131ec47083e admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:33.012258 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafcf20-d854-4df6-b6b4-2131ec47083e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:33.012530 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafcf20-d854-4df6-b6b4-2131ec47083e admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:33.013178 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaafcf20-d854-4df6-b6b4-2131ec47083e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:16:33.013546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaafcf20-d854-4df6-b6b4-2131ec47083e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:33.015289 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaafcf20-d854-4df6-b6b4-2131ec47083e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:33.030822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1012/2041] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 17:16:33.032172 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaafcf20-d854-4df6-b6b4-2131ec47083e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:33.042885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaafcf20-d854-4df6-b6b4-2131ec47083e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:33.047739 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb9fb0-23ef-412e-808e-7a0a0d9fa0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:33.048340 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb9fb0-23ef-412e-808e-7a0a0d9fa0e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:33.048627 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb9fb0-23ef-412e-808e-7a0a0d9fa0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:33.048897 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb9fb0-23ef-412e-808e-7a0a0d9fa0e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:33.049534 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb9fb0-23ef-412e-808e-7a0a0d9fa0e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:33.051470 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb9fb0-23ef-412e-808e-7a0a0d9fa0e1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:33.213153 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1013/2042] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:16:33 2022] POST /identity/v3/projects => generated 379 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:16:33.217360 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb9fb0-23ef-412e-808e-7a0a0d9fa0e1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:33.225368 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-783dde2a-1221-4345-b0ff-c24ae1a165d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '210be08699fb4653a32b6538aae2ebff'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:33.286217 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb9fb0-23ef-412e-808e-7a0a0d9fa0e1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:33.291089 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bae052-80f4-4872-a045-e79ce712dd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:33.291699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bae052-80f4-4872-a045-e79ce712dd75 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:33.291982 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bae052-80f4-4872-a045-e79ce712dd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:33.292254 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bae052-80f4-4872-a045-e79ce712dd75 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:33.292890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bae052-80f4-4872-a045-e79ce712dd75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:33.294919 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bae052-80f4-4872-a045-e79ce712dd75 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:33.423019 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-783dde2a-1221-4345-b0ff-c24ae1a165d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '210be08699fb4653a32b6538aae2ebff'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:33.431702 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1014/2043] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:16:33 2022] POST /identity/v3/users => generated 290 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:16:33.433759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-783dde2a-1221-4345-b0ff-c24ae1a165d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '210be08699fb4653a32b6538aae2ebff'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:33.436299 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bae052-80f4-4872-a045-e79ce712dd75 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:33.449387 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-783dde2a-1221-4345-b0ff-c24ae1a165d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '210be08699fb4653a32b6538aae2ebff'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:33.450384 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-783dde2a-1221-4345-b0ff-c24ae1a165d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '210be08699fb4653a32b6538aae2ebff'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:33.457179 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1030/2044] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:16:32 2022] DELETE /identity/v3/users/210be08699fb4653a32b6538aae2ebff => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:16:33.459035 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bae052-80f4-4872-a045-e79ce712dd75 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:33.463746 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e198117-8042-4284-b1fe-5d492183d541 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:33.464135 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e198117-8042-4284-b1fe-5d492183d541 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:33.464483 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e198117-8042-4284-b1fe-5d492183d541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:33.485880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7e198117-8042-4284-b1fe-5d492183d541 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:33.487469 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7e198117-8042-4284-b1fe-5d492183d541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:33.553580 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf142499-5b1f-464a-8a44-98b9e67dea79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:33.554468 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf142499-5b1f-464a-8a44-98b9e67dea79 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:33.555606 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf142499-5b1f-464a-8a44-98b9e67dea79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:33.556015 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf142499-5b1f-464a-8a44-98b9e67dea79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:33.556931 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf142499-5b1f-464a-8a44-98b9e67dea79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:33.559526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf142499-5b1f-464a-8a44-98b9e67dea79 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:33.573680 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1015/2045] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:16:33.574864 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf142499-5b1f-464a-8a44-98b9e67dea79 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:33.576768 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bf142499-5b1f-464a-8a44-98b9e67dea79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:33.588678 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1031/2046] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:16:33.591076 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf142499-5b1f-464a-8a44-98b9e67dea79 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:33.599309 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e198117-8042-4284-b1fe-5d492183d541 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:33.600255 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd19d8e-f45a-4e4f-8fa4-43e48e7f45cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:33.601871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd19d8e-f45a-4e4f-8fa4-43e48e7f45cd admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:33.602365 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd19d8e-f45a-4e4f-8fa4-43e48e7f45cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:33.602733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd19d8e-f45a-4e4f-8fa4-43e48e7f45cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:33.603956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd19d8e-f45a-4e4f-8fa4-43e48e7f45cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:33.605800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd19d8e-f45a-4e4f-8fa4-43e48e7f45cd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:33.612873 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7e198117-8042-4284-b1fe-5d492183d541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:33.619975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1016/2047] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:16:33.624417 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd19d8e-f45a-4e4f-8fa4-43e48e7f45cd admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:33.636198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd19d8e-f45a-4e4f-8fa4-43e48e7f45cd admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:33.665110 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b7cf7-41c4-4d49-b247-8fe02011c4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:33.666071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b7cf7-41c4-4d49-b247-8fe02011c4b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:33.666695 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b7cf7-41c4-4d49-b247-8fe02011c4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:33.667664 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b7cf7-41c4-4d49-b247-8fe02011c4b7 admin admin] PATH_INFO: `/v3/projects/e1bc63782f1d4b32ab58852232df16ac/users/22a5224313e04d35af7fc7802ef5d6ed/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:33.698478 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e198117-8042-4284-b1fe-5d492183d541 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:33.703349 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3b7cf7-41c4-4d49-b247-8fe02011c4b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1bc63782f1d4b32ab58852232df16ac, user_id=22a5224313e04d35af7fc7802ef5d6ed, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:33.704601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96823125-e769-4976-8edb-0071d56a6be8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:33.705250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96823125-e769-4976-8edb-0071d56a6be8 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:33.705529 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96823125-e769-4976-8edb-0071d56a6be8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:33.705943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96823125-e769-4976-8edb-0071d56a6be8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:33.707442 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3b7cf7-41c4-4d49-b247-8fe02011c4b7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:33.717453 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96823125-e769-4976-8edb-0071d56a6be8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:33.719384 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96823125-e769-4976-8edb-0071d56a6be8 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:33.732690 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1032/2048] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:16:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:16:33.847962 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1017/2049] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:16:33 2022] PUT /identity/v3/projects/e1bc63782f1d4b32ab58852232df16ac/users/22a5224313e04d35af7fc7802ef5d6ed/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:16:33.853850 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759c9d3-71e0-4088-b614-12c47ef0e706 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:33.854377 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759c9d3-71e0-4088-b614-12c47ef0e706 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:33.854377 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759c9d3-71e0-4088-b614-12c47ef0e706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:33.882476 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c759c9d3-71e0-4088-b614-12c47ef0e706 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:33.884340 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c759c9d3-71e0-4088-b614-12c47ef0e706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:33.956451 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1033/2050] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:16:33.962271 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:34.020023 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b7cf7-41c4-4d49-b247-8fe02011c4b7 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:34.023759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3d3b7cf7-41c4-4d49-b247-8fe02011c4b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:34.055955 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:34.103371 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b7cf7-41c4-4d49-b247-8fe02011c4b7 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:34.107825 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba5e243-7514-461c-8038-4781ead6518e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:34.108399 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5e243-7514-461c-8038-4781ead6518e service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:34.109414 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5e243-7514-461c-8038-4781ead6518e service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:34.109414 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5e243-7514-461c-8038-4781ead6518e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:34.137229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba5e243-7514-461c-8038-4781ead6518e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:34.139906 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba5e243-7514-461c-8038-4781ead6518e service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:34.154104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1018/2051] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:16:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:16:34.415113 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759c9d3-71e0-4088-b614-12c47ef0e706 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:34.416587 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c759c9d3-71e0-4088-b614-12c47ef0e706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:34.452482 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759c9d3-71e0-4088-b614-12c47ef0e706 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:34.457958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a20fd6-55c5-435d-8729-a925a14257c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:34.458655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a20fd6-55c5-435d-8729-a925a14257c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:34.458932 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a20fd6-55c5-435d-8729-a925a14257c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:34.459200 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a20fd6-55c5-435d-8729-a925a14257c8 admin admin] PATH_INFO: `/v3/projects/201437c0acd2463cb2f09424dfc6f085` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:34.460360 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a20fd6-55c5-435d-8729-a925a14257c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=201437c0acd2463cb2f09424dfc6f085)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:34.463018 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a20fd6-55c5-435d-8729-a925a14257c8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:34.628767 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d4a20fd6-55c5-435d-8729-a925a14257c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '201437c0acd2463cb2f09424dfc6f085', 'request_id': 'req-d4a20fd6-55c5-435d-8729-a925a14257c8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:34.637764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d4a20fd6-55c5-435d-8729-a925a14257c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '201437c0acd2463cb2f09424dfc6f085', 'request_id': 'req-d4a20fd6-55c5-435d-8729-a925a14257c8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:34.708363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d4a20fd6-55c5-435d-8729-a925a14257c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '201437c0acd2463cb2f09424dfc6f085', 'request_id': 'req-d4a20fd6-55c5-435d-8729-a925a14257c8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:34.723325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d4a20fd6-55c5-435d-8729-a925a14257c8 admin admin] The token cache is being invalidate because project 201437c0acd2463cb2f09424dfc6f085 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:16:34.723612 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d4a20fd6-55c5-435d-8729-a925a14257c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:34.724283 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d4a20fd6-55c5-435d-8729-a925a14257c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:34.725938 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1034/2052] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:16:34 2022] DELETE /identity/v3/projects/201437c0acd2463cb2f09424dfc6f085 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:16:40.321415 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f994580-9511-4ebc-8e3b-4388721d1f58 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:40.321991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f994580-9511-4ebc-8e3b-4388721d1f58 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:40.321991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f994580-9511-4ebc-8e3b-4388721d1f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:40.359902 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3f994580-9511-4ebc-8e3b-4388721d1f58 None None] MFA Rules not processed for user `22a5224313e04d35af7fc7802ef5d6ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:40.361108 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3f994580-9511-4ebc-8e3b-4388721d1f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:40.457435 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1019/2053] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:16:40.462959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a20fd6-55c5-435d-8729-a925a14257c8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:40.464455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d4a20fd6-55c5-435d-8729-a925a14257c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:40.542927 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a20fd6-55c5-435d-8729-a925a14257c8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:40.547910 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5048ee-fd11-4575-877d-d74b057148a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:40.548598 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5048ee-fd11-4575-877d-d74b057148a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:40.548882 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5048ee-fd11-4575-877d-d74b057148a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:40.549151 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5048ee-fd11-4575-877d-d74b057148a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:40.549799 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5048ee-fd11-4575-877d-d74b057148a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:40.551820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5048ee-fd11-4575-877d-d74b057148a2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:40.570091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:40.643825 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1035/2054] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:16:40 2022] POST /identity/v3/projects => generated 379 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:16:40.648978 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f994580-9511-4ebc-8e3b-4388721d1f58 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:40.660993 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f994580-9511-4ebc-8e3b-4388721d1f58 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:40.746660 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b3a190-8d68-4eca-90ed-3c12378aeebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:40.747287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3a190-8d68-4eca-90ed-3c12378aeebf admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:40.747732 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3a190-8d68-4eca-90ed-3c12378aeebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:40.747992 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3a190-8d68-4eca-90ed-3c12378aeebf admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:40.748627 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b3a190-8d68-4eca-90ed-3c12378aeebf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:40.750614 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b3a190-8d68-4eca-90ed-3c12378aeebf admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:40.761935 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:40.875763 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1020/2055] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:16:40 2022] POST /identity/v3/users => generated 296 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:16:40.882172 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5048ee-fd11-4575-877d-d74b057148a2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:40.894620 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5048ee-fd11-4575-877d-d74b057148a2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:40.900925 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365c7c0-e26c-4981-8cc3-96037f0bdf5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:40.901563 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365c7c0-e26c-4981-8cc3-96037f0bdf5c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:40.901818 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365c7c0-e26c-4981-8cc3-96037f0bdf5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:40.902086 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365c7c0-e26c-4981-8cc3-96037f0bdf5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:40.907979 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b365c7c0-e26c-4981-8cc3-96037f0bdf5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:40.907979 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b365c7c0-e26c-4981-8cc3-96037f0bdf5c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:40.920860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1036/2056] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:16:40.926243 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b3a190-8d68-4eca-90ed-3c12378aeebf admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:40.938541 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b3a190-8d68-4eca-90ed-3c12378aeebf admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:40.943805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ca636-863e-4bd6-a5c0-ee971837c299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:40.944528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ca636-863e-4bd6-a5c0-ee971837c299 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:40.944766 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ca636-863e-4bd6-a5c0-ee971837c299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:40.945583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ca636-863e-4bd6-a5c0-ee971837c299 admin admin] PATH_INFO: `/v3/projects/ef4319031106499c8cd532dfce92cc89/users/b21eaa7c53f64f2a94e29f9b24efaef9/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:40.965658 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861ca636-863e-4bd6-a5c0-ee971837c299 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef4319031106499c8cd532dfce92cc89, user_id=b21eaa7c53f64f2a94e29f9b24efaef9, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:40.968392 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861ca636-863e-4bd6-a5c0-ee971837c299 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:41.079277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1021/2057] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:16:40 2022] PUT /identity/v3/projects/ef4319031106499c8cd532dfce92cc89/users/b21eaa7c53f64f2a94e29f9b24efaef9/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:16:41.083370 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365c7c0-e26c-4981-8cc3-96037f0bdf5c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:41.111384 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365c7c0-e26c-4981-8cc3-96037f0bdf5c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:41.214248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07aedd-fc17-4c38-970a-2982c51dedda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:41.214954 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07aedd-fc17-4c38-970a-2982c51dedda admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:41.215238 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07aedd-fc17-4c38-970a-2982c51dedda admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:41.215509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07aedd-fc17-4c38-970a-2982c51dedda admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:41.216208 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e07aedd-fc17-4c38-970a-2982c51dedda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:41.218060 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e07aedd-fc17-4c38-970a-2982c51dedda admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:41.234988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1037/2058] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:16:41.239168 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ca636-863e-4bd6-a5c0-ee971837c299 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:41.263170 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ca636-863e-4bd6-a5c0-ee971837c299 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:41.271687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba21ff0-dd37-4096-b0b1-1af8b43b1c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:41.272399 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba21ff0-dd37-4096-b0b1-1af8b43b1c82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:41.272673 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba21ff0-dd37-4096-b0b1-1af8b43b1c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:41.272945 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba21ff0-dd37-4096-b0b1-1af8b43b1c82 admin admin] PATH_INFO: `/v3/projects/ef4319031106499c8cd532dfce92cc89/users/b21eaa7c53f64f2a94e29f9b24efaef9/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:41.280665 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba21ff0-dd37-4096-b0b1-1af8b43b1c82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef4319031106499c8cd532dfce92cc89, user_id=b21eaa7c53f64f2a94e29f9b24efaef9, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:41.283616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba21ff0-dd37-4096-b0b1-1af8b43b1c82 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:41.302404 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1022/2059] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:16:41 2022] PUT /identity/v3/projects/ef4319031106499c8cd532dfce92cc89/users/b21eaa7c53f64f2a94e29f9b24efaef9/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:16:44.932740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07aedd-fc17-4c38-970a-2982c51dedda admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:44.934833 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8e07aedd-fc17-4c38-970a-2982c51dedda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:44.951406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07aedd-fc17-4c38-970a-2982c51dedda admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:45.118448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3442e-166d-43cc-af87-214a592cdee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:45.119429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3442e-166d-43cc-af87-214a592cdee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:45.120029 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3442e-166d-43cc-af87-214a592cdee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:45.120753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3442e-166d-43cc-af87-214a592cdee4 admin admin] PATH_INFO: `/v3/users/7cdb7160092f4381890bb5a0cb2964ac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:45.122522 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb3442e-166d-43cc-af87-214a592cdee4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cdb7160092f4381890bb5a0cb2964ac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:45.125319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb3442e-166d-43cc-af87-214a592cdee4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:45.390451 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-efb3442e-166d-43cc-af87-214a592cdee4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cdb7160092f4381890bb5a0cb2964ac'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:45.473790 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-efb3442e-166d-43cc-af87-214a592cdee4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cdb7160092f4381890bb5a0cb2964ac'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:45.485722 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-efb3442e-166d-43cc-af87-214a592cdee4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cdb7160092f4381890bb5a0cb2964ac'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:45.535511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-efb3442e-166d-43cc-af87-214a592cdee4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cdb7160092f4381890bb5a0cb2964ac'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:45.541816 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-efb3442e-166d-43cc-af87-214a592cdee4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cdb7160092f4381890bb5a0cb2964ac'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:45.541816 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:45.553270 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1038/2060] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:16:44 2022] DELETE /identity/v3/users/7cdb7160092f4381890bb5a0cb2964ac => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:16:45.555558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9ae46-45bb-439d-bbe8-4f1ffeaa99ec None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:45.556434 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9ae46-45bb-439d-bbe8-4f1ffeaa99ec None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:45.556885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9ae46-45bb-439d-bbe8-4f1ffeaa99ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:45.614204 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a2c9ae46-45bb-439d-bbe8-4f1ffeaa99ec None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:45.619291 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a2c9ae46-45bb-439d-bbe8-4f1ffeaa99ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:45.769079 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1023/2061] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:16:45.778880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3442e-166d-43cc-af87-214a592cdee4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:45.780923 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-efb3442e-166d-43cc-af87-214a592cdee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:45.921626 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3442e-166d-43cc-af87-214a592cdee4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:45.926847 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ffcb7-63a4-49cc-b1db-fa4a78e29f49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:45.927470 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ffcb7-63a4-49cc-b1db-fa4a78e29f49 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:45.927751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ffcb7-63a4-49cc-b1db-fa4a78e29f49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:45.928020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ffcb7-63a4-49cc-b1db-fa4a78e29f49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:45.939359 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ffcb7-63a4-49cc-b1db-fa4a78e29f49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:45.940838 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ffcb7-63a4-49cc-b1db-fa4a78e29f49 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:45.951506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1039/2062] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:16:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:16:46.596967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9ae46-45bb-439d-bbe8-4f1ffeaa99ec None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:46.596967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a2c9ae46-45bb-439d-bbe8-4f1ffeaa99ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:46.641071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9ae46-45bb-439d-bbe8-4f1ffeaa99ec None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:46.655804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c204737-6d6e-4d88-b542-8d9dc1afc360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:46.656916 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c204737-6d6e-4d88-b542-8d9dc1afc360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:46.657341 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c204737-6d6e-4d88-b542-8d9dc1afc360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:46.659553 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c204737-6d6e-4d88-b542-8d9dc1afc360 admin admin] PATH_INFO: `/v3/projects/b389ea9dd5ae4993b48424b0ac24c1af` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:46.661149 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c204737-6d6e-4d88-b542-8d9dc1afc360 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b389ea9dd5ae4993b48424b0ac24c1af)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:46.664425 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c204737-6d6e-4d88-b542-8d9dc1afc360 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:46.796159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2c204737-6d6e-4d88-b542-8d9dc1afc360 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b389ea9dd5ae4993b48424b0ac24c1af', 'request_id': 'req-2c204737-6d6e-4d88-b542-8d9dc1afc360'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:46.942189 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2c204737-6d6e-4d88-b542-8d9dc1afc360 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b389ea9dd5ae4993b48424b0ac24c1af', 'request_id': 'req-2c204737-6d6e-4d88-b542-8d9dc1afc360'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:46.952606 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2c204737-6d6e-4d88-b542-8d9dc1afc360 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b389ea9dd5ae4993b48424b0ac24c1af', 'request_id': 'req-2c204737-6d6e-4d88-b542-8d9dc1afc360'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:46.968858 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2c204737-6d6e-4d88-b542-8d9dc1afc360 admin admin] The token cache is being invalidate because project b389ea9dd5ae4993b48424b0ac24c1af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:16:46.969547 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2c204737-6d6e-4d88-b542-8d9dc1afc360 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:46.970716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2c204737-6d6e-4d88-b542-8d9dc1afc360 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:46.972940 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1024/2063] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:16:46 2022] DELETE /identity/v3/projects/b389ea9dd5ae4993b48424b0ac24c1af => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:16:46.977533 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ffcb7-63a4-49cc-b1db-fa4a78e29f49 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:46.990324 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ffcb7-63a4-49cc-b1db-fa4a78e29f49 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:47.076196 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b93db-aab4-4d22-8d1a-2271f1a88093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:47.078390 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b93db-aab4-4d22-8d1a-2271f1a88093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:47.078909 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b93db-aab4-4d22-8d1a-2271f1a88093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:47.079356 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b93db-aab4-4d22-8d1a-2271f1a88093 admin admin] PATH_INFO: `/v3/users/75a3e1dfb29747c28f3feb2a48329c6b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:47.081068 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b93db-aab4-4d22-8d1a-2271f1a88093 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75a3e1dfb29747c28f3feb2a48329c6b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:47.084965 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b93db-aab4-4d22-8d1a-2271f1a88093 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:47.412661 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2a1b93db-aab4-4d22-8d1a-2271f1a88093 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75a3e1dfb29747c28f3feb2a48329c6b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:47.593402 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2a1b93db-aab4-4d22-8d1a-2271f1a88093 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75a3e1dfb29747c28f3feb2a48329c6b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:47.603732 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2a1b93db-aab4-4d22-8d1a-2271f1a88093 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75a3e1dfb29747c28f3feb2a48329c6b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:47.642918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2a1b93db-aab4-4d22-8d1a-2271f1a88093 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75a3e1dfb29747c28f3feb2a48329c6b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:47.645734 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2a1b93db-aab4-4d22-8d1a-2271f1a88093 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75a3e1dfb29747c28f3feb2a48329c6b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:47.657244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1040/2064] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:16:46 2022] DELETE /identity/v3/users/75a3e1dfb29747c28f3feb2a48329c6b => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:16:48.580331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c204737-6d6e-4d88-b542-8d9dc1afc360 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:48.583381 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2c204737-6d6e-4d88-b542-8d9dc1afc360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:48.602871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c204737-6d6e-4d88-b542-8d9dc1afc360 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:48.680431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:48.714517 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffcacfd-3a3c-4629-b7e4-1bc4b2b5750e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:48.715179 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcacfd-3a3c-4629-b7e4-1bc4b2b5750e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:48.715510 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcacfd-3a3c-4629-b7e4-1bc4b2b5750e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:48.715789 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcacfd-3a3c-4629-b7e4-1bc4b2b5750e admin admin] PATH_INFO: `/v3/projects/f5017fe9f5c44a8cbc9e2a54ca42fd95` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:48.716969 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffcacfd-3a3c-4629-b7e4-1bc4b2b5750e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5017fe9f5c44a8cbc9e2a54ca42fd95)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:48.720166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffcacfd-3a3c-4629-b7e4-1bc4b2b5750e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:48.957799 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6ffcacfd-3a3c-4629-b7e4-1bc4b2b5750e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5017fe9f5c44a8cbc9e2a54ca42fd95', 'request_id': 'req-6ffcacfd-3a3c-4629-b7e4-1bc4b2b5750e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:49.068111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6ffcacfd-3a3c-4629-b7e4-1bc4b2b5750e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5017fe9f5c44a8cbc9e2a54ca42fd95', 'request_id': 'req-6ffcacfd-3a3c-4629-b7e4-1bc4b2b5750e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:49.078204 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6ffcacfd-3a3c-4629-b7e4-1bc4b2b5750e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5017fe9f5c44a8cbc9e2a54ca42fd95', 'request_id': 'req-6ffcacfd-3a3c-4629-b7e4-1bc4b2b5750e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:49.094932 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6ffcacfd-3a3c-4629-b7e4-1bc4b2b5750e admin admin] The token cache is being invalidate because project f5017fe9f5c44a8cbc9e2a54ca42fd95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:16:49.095590 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6ffcacfd-3a3c-4629-b7e4-1bc4b2b5750e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:49.096564 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6ffcacfd-3a3c-4629-b7e4-1bc4b2b5750e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:16:49.098624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1025/2065] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:16:48 2022] DELETE /identity/v3/projects/f5017fe9f5c44a8cbc9e2a54ca42fd95 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:16:49.103797 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d1f38-c652-4e07-bb08-02fe59168b5a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:49.104346 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d1f38-c652-4e07-bb08-02fe59168b5a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:49.104346 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d1f38-c652-4e07-bb08-02fe59168b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:49.131075 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-396d1f38-c652-4e07-bb08-02fe59168b5a None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:49.133822 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-396d1f38-c652-4e07-bb08-02fe59168b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:49.155167 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:49.255850 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1041/2066] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:16:49.264044 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d3571-f077-47b6-8fba-94e322ddb87b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:49.264323 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d3571-f077-47b6-8fba-94e322ddb87b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:49.264858 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d3571-f077-47b6-8fba-94e322ddb87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:49.297047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-9c4d3571-f077-47b6-8fba-94e322ddb87b None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:49.298961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9c4d3571-f077-47b6-8fba-94e322ddb87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:49.309687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1026/2067] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:16:49.314570 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d1f38-c652-4e07-bb08-02fe59168b5a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:49.346187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d1f38-c652-4e07-bb08-02fe59168b5a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:49.360167 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5fb06-e4aa-4c2a-b94d-110e93822a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:49.363306 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5fb06-e4aa-4c2a-b94d-110e93822a3c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:49.363306 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5fb06-e4aa-4c2a-b94d-110e93822a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:49.363306 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5fb06-e4aa-4c2a-b94d-110e93822a3c admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:49.363306 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb5fb06-e4aa-4c2a-b94d-110e93822a3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:16:49.363969 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb5fb06-e4aa-4c2a-b94d-110e93822a3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:49.366532 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb5fb06-e4aa-4c2a-b94d-110e93822a3c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:49.408780 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1042/2068] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:16:49.413112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d3571-f077-47b6-8fba-94e322ddb87b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:49.428560 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d3571-f077-47b6-8fba-94e322ddb87b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:49.437509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c1b21-71f0-4f0f-a573-4928de7d0384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:49.438319 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c1b21-71f0-4f0f-a573-4928de7d0384 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:49.441330 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c1b21-71f0-4f0f-a573-4928de7d0384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:49.441670 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c1b21-71f0-4f0f-a573-4928de7d0384 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:49.444109 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c1b21-71f0-4f0f-a573-4928de7d0384 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:49.448184 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c1b21-71f0-4f0f-a573-4928de7d0384 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:49.520851 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad48c5a-5b24-4104-8b45-10d8b58c88d6 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:49.521498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad48c5a-5b24-4104-8b45-10d8b58c88d6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:49.521780 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad48c5a-5b24-4104-8b45-10d8b58c88d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:49.558948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3ad48c5a-5b24-4104-8b45-10d8b58c88d6 None None] MFA Rules not processed for user `48feb2fc02f04e739c5e1bf7d68baa16`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:49.560499 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3ad48c5a-5b24-4104-8b45-10d8b58c88d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:49.612076 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1027/2069] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:16:49 2022] POST /identity/v3/projects => generated 375 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:16:49.615114 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c1b21-71f0-4f0f-a573-4928de7d0384 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:49.656217 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1043/2070] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:16:49.789833 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c1b21-71f0-4f0f-a573-4928de7d0384 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:49.795700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a12d1a-72ab-4449-af3f-f61f1f8b812f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:49.796624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a12d1a-72ab-4449-af3f-f61f1f8b812f admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:49.797068 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a12d1a-72ab-4449-af3f-f61f1f8b812f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:49.797491 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a12d1a-72ab-4449-af3f-f61f1f8b812f admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:49.798840 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a12d1a-72ab-4449-af3f-f61f1f8b812f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:49.801193 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a12d1a-72ab-4449-af3f-f61f1f8b812f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:49.944239 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1028/2071] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:16:49 2022] POST /identity/v3/users => generated 288 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:16:49.949380 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad48c5a-5b24-4104-8b45-10d8b58c88d6 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:49.967721 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad48c5a-5b24-4104-8b45-10d8b58c88d6 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:49.984555 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e12cd7-d0f4-45e3-9c5e-533ae419c769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:49.986619 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e12cd7-d0f4-45e3-9c5e-533ae419c769 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:49.986619 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e12cd7-d0f4-45e3-9c5e-533ae419c769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:49.986619 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e12cd7-d0f4-45e3-9c5e-533ae419c769 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:49.987347 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e12cd7-d0f4-45e3-9c5e-533ae419c769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:49.991260 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e12cd7-d0f4-45e3-9c5e-533ae419c769 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:50.015102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1044/2072] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:16:50.019074 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a12d1a-72ab-4449-af3f-f61f1f8b812f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:50.038136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a12d1a-72ab-4449-af3f-f61f1f8b812f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:50.044735 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184899f8-ba55-45c2-b11d-cf73837411bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:50.045731 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184899f8-ba55-45c2-b11d-cf73837411bc admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:50.046211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184899f8-ba55-45c2-b11d-cf73837411bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:50.046718 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184899f8-ba55-45c2-b11d-cf73837411bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:50.047663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184899f8-ba55-45c2-b11d-cf73837411bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:50.049551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184899f8-ba55-45c2-b11d-cf73837411bc admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:50.064016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1029/2073] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:16:50.068999 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e12cd7-d0f4-45e3-9c5e-533ae419c769 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:50.082054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e12cd7-d0f4-45e3-9c5e-533ae419c769 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:50.088195 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1429fb-0ebf-4027-9b50-a3e25587983f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:50.089295 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1429fb-0ebf-4027-9b50-a3e25587983f admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:50.089806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1429fb-0ebf-4027-9b50-a3e25587983f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:50.090323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1429fb-0ebf-4027-9b50-a3e25587983f admin admin] PATH_INFO: `/v3/projects/534546ac07094ffa87bd73c681c9947b/users/c458f87e67fb4ef1b297f9e4c4a8e3e2/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:50.109269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1429fb-0ebf-4027-9b50-a3e25587983f admin admin] RBAC: Authorizing `identity:create_grant(project_id=534546ac07094ffa87bd73c681c9947b, user_id=c458f87e67fb4ef1b297f9e4c4a8e3e2, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:50.113003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1429fb-0ebf-4027-9b50-a3e25587983f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:50.176097 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487be036-e99f-4e7e-82db-0e6db27c7a9e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:50.176386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487be036-e99f-4e7e-82db-0e6db27c7a9e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:50.176720 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487be036-e99f-4e7e-82db-0e6db27c7a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:50.187496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1045/2074] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:16:50 2022] PUT /identity/v3/projects/534546ac07094ffa87bd73c681c9947b/users/c458f87e67fb4ef1b297f9e4c4a8e3e2/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:16:50.199461 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51cb32a-1b33-4063-b999-ddfb9de8aa42 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:50.200597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51cb32a-1b33-4063-b999-ddfb9de8aa42 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:50.201076 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51cb32a-1b33-4063-b999-ddfb9de8aa42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:50.223396 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-487be036-e99f-4e7e-82db-0e6db27c7a9e None None] MFA Rules not processed for user `b21eaa7c53f64f2a94e29f9b24efaef9`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:50.227568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-487be036-e99f-4e7e-82db-0e6db27c7a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:50.237979 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d51cb32a-1b33-4063-b999-ddfb9de8aa42 None None] MFA Rules not processed for user `c458f87e67fb4ef1b297f9e4c4a8e3e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:50.240216 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d51cb32a-1b33-4063-b999-ddfb9de8aa42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:50.337222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1030/2075] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:50 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:16:50.350756 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1046/2076] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:16:50.352736 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487be036-e99f-4e7e-82db-0e6db27c7a9e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:50.374942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:50.442575 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487be036-e99f-4e7e-82db-0e6db27c7a9e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:50.448911 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa32b9-14ce-434e-852b-e931f7ba9f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:50.449626 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa32b9-14ce-434e-852b-e931f7ba9f45 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:50.449970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa32b9-14ce-434e-852b-e931f7ba9f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:50.450311 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa32b9-14ce-434e-852b-e931f7ba9f45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:50.451301 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa32b9-14ce-434e-852b-e931f7ba9f45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:50.453354 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa32b9-14ce-434e-852b-e931f7ba9f45 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:50.590543 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1031/2077] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:16:50 2022] POST /identity/v3/projects => generated 377 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:16:50.594865 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51cb32a-1b33-4063-b999-ddfb9de8aa42 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:50.621076 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51cb32a-1b33-4063-b999-ddfb9de8aa42 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:50.628321 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:50.706298 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385771b-d6e9-4945-b625-24d30a6c5f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:50.707059 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385771b-d6e9-4945-b625-24d30a6c5f0c admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:50.707294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385771b-d6e9-4945-b625-24d30a6c5f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:50.707567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385771b-d6e9-4945-b625-24d30a6c5f0c admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:50.708231 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385771b-d6e9-4945-b625-24d30a6c5f0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:50.711049 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385771b-d6e9-4945-b625-24d30a6c5f0c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:50.770816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa32b9-14ce-434e-852b-e931f7ba9f45 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:50.773374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-19fa32b9-14ce-434e-852b-e931f7ba9f45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:50.837838 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1047/2078] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:16:50 2022] POST /identity/v3/users => generated 295 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:16:50.839134 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385771b-d6e9-4945-b625-24d30a6c5f0c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:50.849626 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385771b-d6e9-4945-b625-24d30a6c5f0c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:50.855809 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ebd81-1f67-46fe-940e-eb4ac1dcd95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:50.856436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ebd81-1f67-46fe-940e-eb4ac1dcd95a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:50.856713 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ebd81-1f67-46fe-940e-eb4ac1dcd95a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:50.856985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ebd81-1f67-46fe-940e-eb4ac1dcd95a admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:50.857665 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7ebd81-1f67-46fe-940e-eb4ac1dcd95a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:50.859388 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7ebd81-1f67-46fe-940e-eb4ac1dcd95a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:50.859813 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa32b9-14ce-434e-852b-e931f7ba9f45 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:50.865977 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947ae7a-1315-419f-a0c8-d11a4d7d3671 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:50.867380 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947ae7a-1315-419f-a0c8-d11a4d7d3671 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:50.867761 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947ae7a-1315-419f-a0c8-d11a4d7d3671 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:50.868263 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947ae7a-1315-419f-a0c8-d11a4d7d3671 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:50.870288 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7947ae7a-1315-419f-a0c8-d11a4d7d3671 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:50.871283 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1048/2079] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:16:50.876127 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ebd81-1f67-46fe-940e-eb4ac1dcd95a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:50.887495 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ebd81-1f67-46fe-940e-eb4ac1dcd95a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:50.894237 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c1b0e-825b-4218-affd-2869d951363e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:50.897501 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c1b0e-825b-4218-affd-2869d951363e admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:50.897501 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c1b0e-825b-4218-affd-2869d951363e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:50.897501 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c1b0e-825b-4218-affd-2869d951363e admin admin] PATH_INFO: `/v3/projects/cbddcf17e98d4a92a712f5d10418a3b8/users/2f3290a79ba44adca033f43f84055838/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:50.921511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c1b0e-825b-4218-affd-2869d951363e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbddcf17e98d4a92a712f5d10418a3b8, user_id=2f3290a79ba44adca033f43f84055838, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:50.924759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c1b0e-825b-4218-affd-2869d951363e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:50.950572 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947ae7a-1315-419f-a0c8-d11a4d7d3671 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:50.952048 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947ae7a-1315-419f-a0c8-d11a4d7d3671 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:50.961267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1032/2080] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:16:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:16:51.025971 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1049/2081] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:16:50 2022] PUT /identity/v3/projects/cbddcf17e98d4a92a712f5d10418a3b8/users/2f3290a79ba44adca033f43f84055838/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:16:51.030318 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947ae7a-1315-419f-a0c8-d11a4d7d3671 service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:51.041471 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947ae7a-1315-419f-a0c8-d11a4d7d3671 service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:51.117409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527cce5a-46d9-4357-992d-38b33a5f76cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:51.118340 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cce5a-46d9-4357-992d-38b33a5f76cc admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:51.118598 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cce5a-46d9-4357-992d-38b33a5f76cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:51.118825 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cce5a-46d9-4357-992d-38b33a5f76cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:51.119901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527cce5a-46d9-4357-992d-38b33a5f76cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:51.122155 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527cce5a-46d9-4357-992d-38b33a5f76cc admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:51.138478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1033/2082] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:16:51.139775 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c1b0e-825b-4218-affd-2869d951363e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:51.152475 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c1b0e-825b-4218-affd-2869d951363e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:51.157393 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4848d-9c79-43bd-867c-b1ea32c90008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:51.158152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4848d-9c79-43bd-867c-b1ea32c90008 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:51.158464 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4848d-9c79-43bd-867c-b1ea32c90008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:51.158894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4848d-9c79-43bd-867c-b1ea32c90008 admin admin] PATH_INFO: `/v3/projects/cbddcf17e98d4a92a712f5d10418a3b8/users/2f3290a79ba44adca033f43f84055838/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:51.161316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4848d-9c79-43bd-867c-b1ea32c90008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbddcf17e98d4a92a712f5d10418a3b8, user_id=2f3290a79ba44adca033f43f84055838, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:51.166173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4848d-9c79-43bd-867c-b1ea32c90008 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:51.181028 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1050/2083] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:16:51 2022] PUT /identity/v3/projects/cbddcf17e98d4a92a712f5d10418a3b8/users/2f3290a79ba44adca033f43f84055838/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:16:51.197795 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88151173-3bfc-4ba7-9e73-0e73c4cbd756 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:51.207413 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88151173-3bfc-4ba7-9e73-0e73c4cbd756 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:51.207413 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88151173-3bfc-4ba7-9e73-0e73c4cbd756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:51.256381 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-88151173-3bfc-4ba7-9e73-0e73c4cbd756 None None] MFA Rules not processed for user `2f3290a79ba44adca033f43f84055838`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:51.258892 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-88151173-3bfc-4ba7-9e73-0e73c4cbd756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:51.346182 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1034/2084] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:16:51.358970 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4848d-9c79-43bd-867c-b1ea32c90008 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:51.360901 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3cd4848d-9c79-43bd-867c-b1ea32c90008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:51.441028 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4848d-9c79-43bd-867c-b1ea32c90008 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:51.444879 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546b211-0fe0-4354-bc9d-9286ceb3ce3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:51.445490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546b211-0fe0-4354-bc9d-9286ceb3ce3f service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:51.445824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546b211-0fe0-4354-bc9d-9286ceb3ce3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:51.446098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546b211-0fe0-4354-bc9d-9286ceb3ce3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:51.515039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4546b211-0fe0-4354-bc9d-9286ceb3ce3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:51.516369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4546b211-0fe0-4354-bc9d-9286ceb3ce3f service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:51.543378 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1051/2085] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:16:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:16:51.559644 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:58.227424 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88151173-3bfc-4ba7-9e73-0e73c4cbd756 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:16:58.228910 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-88151173-3bfc-4ba7-9e73-0e73c4cbd756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:58.243834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88151173-3bfc-4ba7-9e73-0e73c4cbd756 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:16:58.247681 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c1f1e-719f-4178-b8bd-6e3d44dc57cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:16:58.248291 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c1f1e-719f-4178-b8bd-6e3d44dc57cb service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:58.248567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c1f1e-719f-4178-b8bd-6e3d44dc57cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:58.248837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c1f1e-719f-4178-b8bd-6e3d44dc57cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:58.259508 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c1f1e-719f-4178-b8bd-6e3d44dc57cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:16:58.260944 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c1f1e-719f-4178-b8bd-6e3d44dc57cb service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:16:58.275332 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1035/2086] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:16:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:16:58.826460 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96763d7f-6e1f-446c-9537-d5029d223afd None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:16:58.827241 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96763d7f-6e1f-446c-9537-d5029d223afd None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:16:58.827671 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96763d7f-6e1f-446c-9537-d5029d223afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:16:58.851461 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:16:58.871142 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-96763d7f-6e1f-446c-9537-d5029d223afd None None] MFA Rules not processed for user `44132cbd18f2470dbe9de6b5125d7db5`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:16:58.872548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-96763d7f-6e1f-446c-9537-d5029d223afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:16:59.003881 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1052/2087] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:16:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:17:01.699435 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f51dd-fd50-4e7f-a590-d2efbb48f5b9 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:01.700421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f51dd-fd50-4e7f-a590-d2efbb48f5b9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:01.700701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f51dd-fd50-4e7f-a590-d2efbb48f5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:01.722317 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7e1f51dd-fd50-4e7f-a590-d2efbb48f5b9 None None] MFA Rules not processed for user `7d2a65984c4f42fbb18977c7345c42ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:17:01.723604 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7e1f51dd-fd50-4e7f-a590-d2efbb48f5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:01.754894 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:17:01.786853 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1036/2088] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:17:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:17:09.250529 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=24 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:17:11.844637 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=10 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:17:18.555195 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e17379-06d3-40ee-b477-388bd45df4fc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:18.562586 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e17379-06d3-40ee-b477-388bd45df4fc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:18.562586 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e17379-06d3-40ee-b477-388bd45df4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:18.579849 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-22e17379-06d3-40ee-b477-388bd45df4fc None None] MFA Rules not processed for user `22a5224313e04d35af7fc7802ef5d6ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:17:18.586764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-22e17379-06d3-40ee-b477-388bd45df4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:18.700897 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1053/2089] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:17:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:17:25.433538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f51dd-fd50-4e7f-a590-d2efbb48f5b9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:25.435151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7e1f51dd-fd50-4e7f-a590-d2efbb48f5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:25.528101 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f51dd-fd50-4e7f-a590-d2efbb48f5b9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:25.533973 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a0507-fe36-4dfc-8316-ac6e72ae96c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cd39d9a583a4d73a5654017be106019', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:25.534640 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a0507-fe36-4dfc-8316-ac6e72ae96c3 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:25.534920 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a0507-fe36-4dfc-8316-ac6e72ae96c3 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:25.536069 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a0507-fe36-4dfc-8316-ac6e72ae96c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:25.537066 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a22a0507-fe36-4dfc-8316-ac6e72ae96c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:25.555097 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a0507-fe36-4dfc-8316-ac6e72ae96c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:25.558296 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a0507-fe36-4dfc-8316-ac6e72ae96c3 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:25.573125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1037/2090] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:17:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:17:26.237817 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629cdd5-0805-429b-a831-05d2ece8cb85 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:26.238417 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629cdd5-0805-429b-a831-05d2ece8cb85 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:26.238794 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629cdd5-0805-429b-a831-05d2ece8cb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:26.261244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e629cdd5-0805-429b-a831-05d2ece8cb85 None None] MFA Rules not processed for user `4f9e1767ec90435f80248a2930081265`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:17:26.262659 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e629cdd5-0805-429b-a831-05d2ece8cb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:26.368564 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1054/2091] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 10 17:17:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:17:26.423455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a0507-fe36-4dfc-8316-ac6e72ae96c3 service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:26.424914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a22a0507-fe36-4dfc-8316-ac6e72ae96c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:26.512963 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a0507-fe36-4dfc-8316-ac6e72ae96c3 service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:26.520315 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d6a4d-d52e-4ab4-bcaf-b1d8b79c6423 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4986821ef9f84b99a835e387635aa9a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:26.520990 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d6a4d-d52e-4ab4-bcaf-b1d8b79c6423 service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:26.521446 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d6a4d-d52e-4ab4-bcaf-b1d8b79c6423 service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:26.521716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d6a4d-d52e-4ab4-bcaf-b1d8b79c6423 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:26.534056 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378d6a4d-d52e-4ab4-bcaf-b1d8b79c6423 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:26.541315 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378d6a4d-d52e-4ab4-bcaf-b1d8b79c6423 service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:26.555993 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1038/2092] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 10 17:17:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 17:17:29.773589 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629cdd5-0805-429b-a831-05d2ece8cb85 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:29.859692 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629cdd5-0805-429b-a831-05d2ece8cb85 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:29.865193 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db8db0-eae3-4b38-b281-c4defc1ad5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:29.866574 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db8db0-eae3-4b38-b281-c4defc1ad5bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:29.867049 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db8db0-eae3-4b38-b281-c4defc1ad5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:29.867416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db8db0-eae3-4b38-b281-c4defc1ad5bd admin admin] PATH_INFO: `/v3/users/c458f87e67fb4ef1b297f9e4c4a8e3e2` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:29.869536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db8db0-eae3-4b38-b281-c4defc1ad5bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c458f87e67fb4ef1b297f9e4c4a8e3e2)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:29.872471 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db8db0-eae3-4b38-b281-c4defc1ad5bd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:30.195736 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f1db8db0-eae3-4b38-b281-c4defc1ad5bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c458f87e67fb4ef1b297f9e4c4a8e3e2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:30.201422 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:17:30.290781 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f1db8db0-eae3-4b38-b281-c4defc1ad5bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c458f87e67fb4ef1b297f9e4c4a8e3e2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:30.300188 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f1db8db0-eae3-4b38-b281-c4defc1ad5bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c458f87e67fb4ef1b297f9e4c4a8e3e2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:30.326626 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f1db8db0-eae3-4b38-b281-c4defc1ad5bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c458f87e67fb4ef1b297f9e4c4a8e3e2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:30.327489 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f1db8db0-eae3-4b38-b281-c4defc1ad5bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c458f87e67fb4ef1b297f9e4c4a8e3e2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:30.341138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1055/2093] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:17:29 2022] DELETE /identity/v3/users/c458f87e67fb4ef1b297f9e4c4a8e3e2 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:17:30.346895 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fdbe6b-64d5-4bac-9367-fd85d2bfaaf1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:30.347729 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fdbe6b-64d5-4bac-9367-fd85d2bfaaf1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:30.348482 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fdbe6b-64d5-4bac-9367-fd85d2bfaaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:30.392732 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b7fdbe6b-64d5-4bac-9367-fd85d2bfaaf1 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:17:30.394276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b7fdbe6b-64d5-4bac-9367-fd85d2bfaaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:30.528956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db8db0-eae3-4b38-b281-c4defc1ad5bd admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:30.531706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f1db8db0-eae3-4b38-b281-c4defc1ad5bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:30.536631 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1039/2094] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:17:30.703741 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db8db0-eae3-4b38-b281-c4defc1ad5bd admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:30.708727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b10e0-eb64-414f-8e60-604537a38eae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:30.709051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b10e0-eb64-414f-8e60-604537a38eae service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:30.709513 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b10e0-eb64-414f-8e60-604537a38eae service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:30.710938 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b10e0-eb64-414f-8e60-604537a38eae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:30.743754 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b10e0-eb64-414f-8e60-604537a38eae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:30.745648 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b10e0-eb64-414f-8e60-604537a38eae service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:30.765609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1056/2095] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:17:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:17:32.032335 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fdbe6b-64d5-4bac-9367-fd85d2bfaaf1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:32.034709 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b7fdbe6b-64d5-4bac-9367-fd85d2bfaaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:32.062804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fdbe6b-64d5-4bac-9367-fd85d2bfaaf1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:32.069934 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c30d8-b272-4869-8ebe-d617152bfefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:32.071130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c30d8-b272-4869-8ebe-d617152bfefc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:32.071835 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c30d8-b272-4869-8ebe-d617152bfefc admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:32.072233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c30d8-b272-4869-8ebe-d617152bfefc admin admin] PATH_INFO: `/v3/projects/534546ac07094ffa87bd73c681c9947b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:32.074001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c30d8-b272-4869-8ebe-d617152bfefc admin admin] RBAC: Authorizing `identity:delete_project(project_id=534546ac07094ffa87bd73c681c9947b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:32.090280 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c30d8-b272-4869-8ebe-d617152bfefc admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:32.114631 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:17:32.275662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4b5c30d8-b272-4869-8ebe-d617152bfefc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '534546ac07094ffa87bd73c681c9947b', 'request_id': 'req-4b5c30d8-b272-4869-8ebe-d617152bfefc'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:32.443113 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4b5c30d8-b272-4869-8ebe-d617152bfefc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '534546ac07094ffa87bd73c681c9947b', 'request_id': 'req-4b5c30d8-b272-4869-8ebe-d617152bfefc'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:32.458818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4b5c30d8-b272-4869-8ebe-d617152bfefc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '534546ac07094ffa87bd73c681c9947b', 'request_id': 'req-4b5c30d8-b272-4869-8ebe-d617152bfefc'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:32.478724 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4b5c30d8-b272-4869-8ebe-d617152bfefc admin admin] The token cache is being invalidate because project 534546ac07094ffa87bd73c681c9947b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:17:32.479019 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4b5c30d8-b272-4869-8ebe-d617152bfefc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:32.483412 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4b5c30d8-b272-4869-8ebe-d617152bfefc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:32.485054 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1040/2096] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:17:32 2022] DELETE /identity/v3/projects/534546ac07094ffa87bd73c681c9947b => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:17:32.489760 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b10e0-eb64-414f-8e60-604537a38eae service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:32.501919 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b10e0-eb64-414f-8e60-604537a38eae service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:32.601484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3402c6-23e1-4c72-9f1a-2d6448f2bf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:32.613234 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3402c6-23e1-4c72-9f1a-2d6448f2bf33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:32.613234 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3402c6-23e1-4c72-9f1a-2d6448f2bf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:32.620242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3402c6-23e1-4c72-9f1a-2d6448f2bf33 admin admin] PATH_INFO: `/v3/users/2f3290a79ba44adca033f43f84055838` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:32.622139 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3402c6-23e1-4c72-9f1a-2d6448f2bf33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f3290a79ba44adca033f43f84055838)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:32.624378 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3402c6-23e1-4c72-9f1a-2d6448f2bf33 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:32.906770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1c3402c6-23e1-4c72-9f1a-2d6448f2bf33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f3290a79ba44adca033f43f84055838'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:32.993705 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1c3402c6-23e1-4c72-9f1a-2d6448f2bf33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f3290a79ba44adca033f43f84055838'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:33.007336 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1c3402c6-23e1-4c72-9f1a-2d6448f2bf33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f3290a79ba44adca033f43f84055838'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:33.042068 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:17:33.051867 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1c3402c6-23e1-4c72-9f1a-2d6448f2bf33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f3290a79ba44adca033f43f84055838'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:33.056428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1c3402c6-23e1-4c72-9f1a-2d6448f2bf33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f3290a79ba44adca033f43f84055838'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:33.069472 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1057/2097] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:17:32 2022] DELETE /identity/v3/users/2f3290a79ba44adca033f43f84055838 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:17:33.786249 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c30d8-b272-4869-8ebe-d617152bfefc admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:33.792336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4b5c30d8-b272-4869-8ebe-d617152bfefc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:33.810970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c30d8-b272-4869-8ebe-d617152bfefc admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:33.918459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f91fa2-a35d-49fe-b95b-2d75210e3803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:33.919153 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f91fa2-a35d-49fe-b95b-2d75210e3803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:33.919494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f91fa2-a35d-49fe-b95b-2d75210e3803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:33.919775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f91fa2-a35d-49fe-b95b-2d75210e3803 admin admin] PATH_INFO: `/v3/projects/cbddcf17e98d4a92a712f5d10418a3b8` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:33.921141 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f91fa2-a35d-49fe-b95b-2d75210e3803 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbddcf17e98d4a92a712f5d10418a3b8)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:33.923694 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f91fa2-a35d-49fe-b95b-2d75210e3803 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:34.096953 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c6f91fa2-a35d-49fe-b95b-2d75210e3803 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbddcf17e98d4a92a712f5d10418a3b8', 'request_id': 'req-c6f91fa2-a35d-49fe-b95b-2d75210e3803'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:34.181211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c6f91fa2-a35d-49fe-b95b-2d75210e3803 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbddcf17e98d4a92a712f5d10418a3b8', 'request_id': 'req-c6f91fa2-a35d-49fe-b95b-2d75210e3803'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:34.191290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c6f91fa2-a35d-49fe-b95b-2d75210e3803 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbddcf17e98d4a92a712f5d10418a3b8', 'request_id': 'req-c6f91fa2-a35d-49fe-b95b-2d75210e3803'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:34.204955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c6f91fa2-a35d-49fe-b95b-2d75210e3803 admin admin] The token cache is being invalidate because project cbddcf17e98d4a92a712f5d10418a3b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:17:34.205250 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c6f91fa2-a35d-49fe-b95b-2d75210e3803 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:34.205928 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c6f91fa2-a35d-49fe-b95b-2d75210e3803 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:34.210395 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1041/2098] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:17:33 2022] DELETE /identity/v3/projects/cbddcf17e98d4a92a712f5d10418a3b8 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:17:34.213641 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c375e-98f8-4ac1-af34-f36f79eecc05 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:34.215022 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c375e-98f8-4ac1-af34-f36f79eecc05 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:34.215463 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c375e-98f8-4ac1-af34-f36f79eecc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:34.240335 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-652c375e-98f8-4ac1-af34-f36f79eecc05 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:17:34.243264 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-652c375e-98f8-4ac1-af34-f36f79eecc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:34.360771 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1058/2099] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:17:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:17:34.367196 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b22c65-94b4-4dcf-aff1-9d62157cd727 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:34.367824 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b22c65-94b4-4dcf-aff1-9d62157cd727 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:34.368270 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b22c65-94b4-4dcf-aff1-9d62157cd727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:34.395697 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f8b22c65-94b4-4dcf-aff1-9d62157cd727 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:17:34.400303 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f8b22c65-94b4-4dcf-aff1-9d62157cd727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:34.414732 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1042/2100] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:17:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:17:34.419556 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c375e-98f8-4ac1-af34-f36f79eecc05 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:34.431625 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c375e-98f8-4ac1-af34-f36f79eecc05 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:34.438189 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc25406-ba10-4197-aa8a-a2712657d2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:34.439711 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc25406-ba10-4197-aa8a-a2712657d2f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:34.440000 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc25406-ba10-4197-aa8a-a2712657d2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:34.440267 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc25406-ba10-4197-aa8a-a2712657d2f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:34.440933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc25406-ba10-4197-aa8a-a2712657d2f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:17:34.441230 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc25406-ba10-4197-aa8a-a2712657d2f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:34.443174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc25406-ba10-4197-aa8a-a2712657d2f5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:34.470024 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1059/2101] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:17:34.474552 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b22c65-94b4-4dcf-aff1-9d62157cd727 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:34.497894 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b22c65-94b4-4dcf-aff1-9d62157cd727 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:34.507184 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867bc2d4-ed43-4dc2-a1e8-f2c189042194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:34.508061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bc2d4-ed43-4dc2-a1e8-f2c189042194 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:34.508318 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bc2d4-ed43-4dc2-a1e8-f2c189042194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:34.508585 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bc2d4-ed43-4dc2-a1e8-f2c189042194 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:34.509246 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867bc2d4-ed43-4dc2-a1e8-f2c189042194 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:34.512292 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867bc2d4-ed43-4dc2-a1e8-f2c189042194 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:35.292539 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1043/2102] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:17:34 2022] POST /identity/v3/projects => generated 381 bytes in 819 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:17:35.297042 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc25406-ba10-4197-aa8a-a2712657d2f5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:35.308264 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc25406-ba10-4197-aa8a-a2712657d2f5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:35.373604 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02967b2b-a6a8-40f1-a61c-6a024d63fce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:35.374269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02967b2b-a6a8-40f1-a61c-6a024d63fce0 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:35.374583 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02967b2b-a6a8-40f1-a61c-6a024d63fce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:35.374858 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02967b2b-a6a8-40f1-a61c-6a024d63fce0 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:35.375483 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02967b2b-a6a8-40f1-a61c-6a024d63fce0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:35.377403 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02967b2b-a6a8-40f1-a61c-6a024d63fce0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:35.535187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1060/2103] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:17:35 2022] POST /identity/v3/users => generated 291 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:17:35.539847 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867bc2d4-ed43-4dc2-a1e8-f2c189042194 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:35.548800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:17:35.558045 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867bc2d4-ed43-4dc2-a1e8-f2c189042194 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:35.569959 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326b4aa-0b06-4b0c-9bb8-caa79e779af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:35.570752 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326b4aa-0b06-4b0c-9bb8-caa79e779af4 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:35.570968 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326b4aa-0b06-4b0c-9bb8-caa79e779af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:35.571317 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326b4aa-0b06-4b0c-9bb8-caa79e779af4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:35.571847 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326b4aa-0b06-4b0c-9bb8-caa79e779af4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:35.573626 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326b4aa-0b06-4b0c-9bb8-caa79e779af4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:35.602819 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02967b2b-a6a8-40f1-a61c-6a024d63fce0 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:35.612153 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1044/2104] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:17:35.628872 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02967b2b-a6a8-40f1-a61c-6a024d63fce0 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:35.639430 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173fbede-75e9-4f42-8590-bdd332742a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:35.639913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173fbede-75e9-4f42-8590-bdd332742a2a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:35.640259 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173fbede-75e9-4f42-8590-bdd332742a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:35.640474 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173fbede-75e9-4f42-8590-bdd332742a2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:35.641150 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173fbede-75e9-4f42-8590-bdd332742a2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:35.642987 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173fbede-75e9-4f42-8590-bdd332742a2a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:35.664606 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1061/2105] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:17:35.666546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326b4aa-0b06-4b0c-9bb8-caa79e779af4 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:35.686964 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326b4aa-0b06-4b0c-9bb8-caa79e779af4 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:35.697691 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9281ac-9b71-447a-828e-779f9da1a626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:35.701543 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9281ac-9b71-447a-828e-779f9da1a626 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:35.701983 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9281ac-9b71-447a-828e-779f9da1a626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:35.703307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9281ac-9b71-447a-828e-779f9da1a626 admin admin] PATH_INFO: `/v3/projects/d76bce80e88b4e688ce869fd539b9d00/users/62fdc51ecaa14ae094540a488fa74f5f/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:35.728927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9281ac-9b71-447a-828e-779f9da1a626 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d76bce80e88b4e688ce869fd539b9d00, user_id=62fdc51ecaa14ae094540a488fa74f5f, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:35.733180 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9281ac-9b71-447a-828e-779f9da1a626 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:35.831300 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1045/2106] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:17:35 2022] PUT /identity/v3/projects/d76bce80e88b4e688ce869fd539b9d00/users/62fdc51ecaa14ae094540a488fa74f5f/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:17:35.846327 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b13948-1a6e-48a9-bdbc-6eba23807089 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:35.846626 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b13948-1a6e-48a9-bdbc-6eba23807089 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:35.846913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b13948-1a6e-48a9-bdbc-6eba23807089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:35.884863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-90b13948-1a6e-48a9-bdbc-6eba23807089 None None] MFA Rules not processed for user `62fdc51ecaa14ae094540a488fa74f5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:17:35.886299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-90b13948-1a6e-48a9-bdbc-6eba23807089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:35.950554 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:17:35.978504 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1062/2107] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:17:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:17:35.983253 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9281ac-9b71-447a-828e-779f9da1a626 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:36.073880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9281ac-9b71-447a-828e-779f9da1a626 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:36.081440 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76195ddf-0b71-4e8a-99c9-17d97c84a5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:36.082386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76195ddf-0b71-4e8a-99c9-17d97c84a5bc admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:36.082386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76195ddf-0b71-4e8a-99c9-17d97c84a5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:36.100624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76195ddf-0b71-4e8a-99c9-17d97c84a5bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:36.100624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76195ddf-0b71-4e8a-99c9-17d97c84a5bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:36.100624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76195ddf-0b71-4e8a-99c9-17d97c84a5bc admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:36.282404 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1046/2108] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:17:35 2022] POST /identity/v3/projects => generated 379 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:17:36.289392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b13948-1a6e-48a9-bdbc-6eba23807089 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:36.304457 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b13948-1a6e-48a9-bdbc-6eba23807089 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:36.401777 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37eead2-a163-4d7c-b339-64c33b269d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:36.402909 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37eead2-a163-4d7c-b339-64c33b269d96 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:36.403309 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37eead2-a163-4d7c-b339-64c33b269d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:36.403673 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37eead2-a163-4d7c-b339-64c33b269d96 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:36.404527 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37eead2-a163-4d7c-b339-64c33b269d96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:36.407460 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37eead2-a163-4d7c-b339-64c33b269d96 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:36.621948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1063/2109] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:17:36 2022] POST /identity/v3/users => generated 296 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:17:36.626749 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76195ddf-0b71-4e8a-99c9-17d97c84a5bc admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:36.649772 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76195ddf-0b71-4e8a-99c9-17d97c84a5bc admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:36.656549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d718037-889d-464b-bef0-a40b4aed053c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:36.667038 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d718037-889d-464b-bef0-a40b4aed053c admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:36.667038 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d718037-889d-464b-bef0-a40b4aed053c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:36.667038 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d718037-889d-464b-bef0-a40b4aed053c admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:36.667038 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d718037-889d-464b-bef0-a40b4aed053c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:36.667038 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d718037-889d-464b-bef0-a40b4aed053c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:36.677178 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1047/2110] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:17:36.683079 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37eead2-a163-4d7c-b339-64c33b269d96 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:36.699594 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37eead2-a163-4d7c-b339-64c33b269d96 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:36.705800 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab35d0-a8b5-49e1-b54b-c6e3f83cd298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:36.707007 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab35d0-a8b5-49e1-b54b-c6e3f83cd298 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:36.707533 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab35d0-a8b5-49e1-b54b-c6e3f83cd298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:36.708085 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab35d0-a8b5-49e1-b54b-c6e3f83cd298 admin admin] PATH_INFO: `/v3/projects/1805b8e7a86148f28f5da9748b61b7d0/users/9ec4dfbd2b184ca4b6c1ef3f807d545d/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:36.727392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab35d0-a8b5-49e1-b54b-c6e3f83cd298 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1805b8e7a86148f28f5da9748b61b7d0, user_id=9ec4dfbd2b184ca4b6c1ef3f807d545d, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:36.730337 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab35d0-a8b5-49e1-b54b-c6e3f83cd298 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:36.840852 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1064/2111] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:17:36 2022] PUT /identity/v3/projects/1805b8e7a86148f28f5da9748b61b7d0/users/9ec4dfbd2b184ca4b6c1ef3f807d545d/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:17:36.847455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d718037-889d-464b-bef0-a40b4aed053c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:36.862439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d718037-889d-464b-bef0-a40b4aed053c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:36.951886 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1906a8d7-ad2c-41da-9ad0-02bc8d1a9210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:36.952788 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906a8d7-ad2c-41da-9ad0-02bc8d1a9210 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:36.953271 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906a8d7-ad2c-41da-9ad0-02bc8d1a9210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:36.953752 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906a8d7-ad2c-41da-9ad0-02bc8d1a9210 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:36.954781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1906a8d7-ad2c-41da-9ad0-02bc8d1a9210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:36.957310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1906a8d7-ad2c-41da-9ad0-02bc8d1a9210 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:36.976290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1048/2112] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:17:36.981138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab35d0-a8b5-49e1-b54b-c6e3f83cd298 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:36.992496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab35d0-a8b5-49e1-b54b-c6e3f83cd298 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:37.001083 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bcfa4-6575-4d51-9adf-1edd2218ad01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:37.035190 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bcfa4-6575-4d51-9adf-1edd2218ad01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:37.035190 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bcfa4-6575-4d51-9adf-1edd2218ad01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:37.035190 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bcfa4-6575-4d51-9adf-1edd2218ad01 admin admin] PATH_INFO: `/v3/projects/1805b8e7a86148f28f5da9748b61b7d0/users/9ec4dfbd2b184ca4b6c1ef3f807d545d/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:37.035190 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84bcfa4-6575-4d51-9adf-1edd2218ad01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1805b8e7a86148f28f5da9748b61b7d0, user_id=9ec4dfbd2b184ca4b6c1ef3f807d545d, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:37.035190 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84bcfa4-6575-4d51-9adf-1edd2218ad01 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:37.035939 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1065/2113] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:17:36 2022] PUT /identity/v3/projects/1805b8e7a86148f28f5da9748b61b7d0/users/9ec4dfbd2b184ca4b6c1ef3f807d545d/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:17:37.057739 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8a78a-2c4e-43ee-af68-19d29a5e1c97 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:37.058422 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8a78a-2c4e-43ee-af68-19d29a5e1c97 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:37.058806 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8a78a-2c4e-43ee-af68-19d29a5e1c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:37.094330 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-05f8a78a-2c4e-43ee-af68-19d29a5e1c97 None None] MFA Rules not processed for user `9ec4dfbd2b184ca4b6c1ef3f807d545d`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:17:37.096429 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-05f8a78a-2c4e-43ee-af68-19d29a5e1c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:37.147149 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:17:37.229492 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1049/2114] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:17:37 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:17:37.244411 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bcfa4-6575-4d51-9adf-1edd2218ad01 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:37.246165 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c84bcfa4-6575-4d51-9adf-1edd2218ad01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:37.328494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bcfa4-6575-4d51-9adf-1edd2218ad01 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:37.336221 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23fbb7-470a-4856-9e2e-81d35e0ad7cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:37.337361 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23fbb7-470a-4856-9e2e-81d35e0ad7cd service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:37.337954 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23fbb7-470a-4856-9e2e-81d35e0ad7cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:37.338700 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23fbb7-470a-4856-9e2e-81d35e0ad7cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:37.353073 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b23fbb7-470a-4856-9e2e-81d35e0ad7cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:37.354896 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b23fbb7-470a-4856-9e2e-81d35e0ad7cd service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:37.371084 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1066/2115] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:17:37 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:17:39.420452 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8a78a-2c4e-43ee-af68-19d29a5e1c97 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:39.433349 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8a78a-2c4e-43ee-af68-19d29a5e1c97 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:39.445233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef5f92-b647-486f-b592-52734f0d461a tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ec4dfbd2b184ca4b6c1ef3f807d545d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1805b8e7a86148f28f5da9748b61b7d0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:39.447857 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef5f92-b647-486f-b592-52734f0d461a tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:39.448352 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef5f92-b647-486f-b592-52734f0d461a tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:39.448699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef5f92-b647-486f-b592-52734f0d461a tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:39.449857 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef5f92-b647-486f-b592-52734f0d461a tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:17:39.458665 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef5f92-b647-486f-b592-52734f0d461a tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:39.461812 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef5f92-b647-486f-b592-52734f0d461a tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:39.483053 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1050/2116] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:17:39.487704 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23fbb7-470a-4856-9e2e-81d35e0ad7cd service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:39.499832 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23fbb7-470a-4856-9e2e-81d35e0ad7cd service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:39.520288 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26a3d9-467e-4936-8c8a-77164f7ec231 tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ec4dfbd2b184ca4b6c1ef3f807d545d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1805b8e7a86148f28f5da9748b61b7d0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:39.521013 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26a3d9-467e-4936-8c8a-77164f7ec231 tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:39.521304 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26a3d9-467e-4936-8c8a-77164f7ec231 tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:39.521575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26a3d9-467e-4936-8c8a-77164f7ec231 tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:39.522597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26a3d9-467e-4936-8c8a-77164f7ec231 tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:39.524603 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26a3d9-467e-4936-8c8a-77164f7ec231 tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:39.677777 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1067/2117] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:17:39 2022] POST /identity/v3/projects => generated 353 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:17:39.682988 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef5f92-b647-486f-b592-52734f0d461a tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:39.700915 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef5f92-b647-486f-b592-52734f0d461a tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:39.820599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b5b1c-2fa2-486f-bcb2-ef01a309b39f tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ec4dfbd2b184ca4b6c1ef3f807d545d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1805b8e7a86148f28f5da9748b61b7d0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:39.822226 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b5b1c-2fa2-486f-bcb2-ef01a309b39f tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:39.822885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b5b1c-2fa2-486f-bcb2-ef01a309b39f tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:39.823772 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b5b1c-2fa2-486f-bcb2-ef01a309b39f tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:39.824977 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b5b1c-2fa2-486f-bcb2-ef01a309b39f tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:17:39.825761 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b5b1c-2fa2-486f-bcb2-ef01a309b39f tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:39.827940 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b5b1c-2fa2-486f-bcb2-ef01a309b39f tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:39.870487 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1051/2118] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:17:39.937397 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26a3d9-467e-4936-8c8a-77164f7ec231 tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:39.958739 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26a3d9-467e-4936-8c8a-77164f7ec231 tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:39.969915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b717890-29bf-41b6-9b5a-547772954a89 tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ec4dfbd2b184ca4b6c1ef3f807d545d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1805b8e7a86148f28f5da9748b61b7d0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:39.971144 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b717890-29bf-41b6-9b5a-547772954a89 tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:39.971689 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b717890-29bf-41b6-9b5a-547772954a89 tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:39.972359 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b717890-29bf-41b6-9b5a-547772954a89 tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] PATH_INFO: `/v3/projects/a8d079a1a63e473a8933b0897ce8bfeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:39.975329 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b717890-29bf-41b6-9b5a-547772954a89 tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a8d079a1a63e473a8933b0897ce8bfeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:39.979413 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b717890-29bf-41b6-9b5a-547772954a89 tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:40.029521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:17:40.186166 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3b717890-29bf-41b6-9b5a-547772954a89 tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8d079a1a63e473a8933b0897ce8bfeb', 'request_id': 'req-3b717890-29bf-41b6-9b5a-547772954a89'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:40.212845 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3b717890-29bf-41b6-9b5a-547772954a89 tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8d079a1a63e473a8933b0897ce8bfeb', 'request_id': 'req-3b717890-29bf-41b6-9b5a-547772954a89'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:40.318096 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3b717890-29bf-41b6-9b5a-547772954a89 tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8d079a1a63e473a8933b0897ce8bfeb', 'request_id': 'req-3b717890-29bf-41b6-9b5a-547772954a89'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:40.337167 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3b717890-29bf-41b6-9b5a-547772954a89 tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] The token cache is being invalidate because project a8d079a1a63e473a8933b0897ce8bfeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:17:40.337516 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3b717890-29bf-41b6-9b5a-547772954a89 tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:40.338587 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3b717890-29bf-41b6-9b5a-547772954a89 tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:40.341692 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1068/2119] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:17:39 2022] DELETE /identity/v3/projects/a8d079a1a63e473a8933b0897ce8bfeb => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:17:41.649812 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b5b1c-2fa2-486f-bcb2-ef01a309b39f tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:41.651580 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0c9b5b1c-2fa2-486f-bcb2-ef01a309b39f tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:41.731473 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b5b1c-2fa2-486f-bcb2-ef01a309b39f tempest-VolumeQuotaClassesTest-380244110 tempest-VolumeQuotaClassesTest-380244110-project-admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:41.786996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:17:41.796954 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11508a2-2f1d-411c-9c31-630c4443fae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:41.797498 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11508a2-2f1d-411c-9c31-630c4443fae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:41.797779 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11508a2-2f1d-411c-9c31-630c4443fae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:41.798049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11508a2-2f1d-411c-9c31-630c4443fae0 admin admin] PATH_INFO: `/v3/users/62fdc51ecaa14ae094540a488fa74f5f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:41.799309 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11508a2-2f1d-411c-9c31-630c4443fae0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62fdc51ecaa14ae094540a488fa74f5f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:41.801351 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11508a2-2f1d-411c-9c31-630c4443fae0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:42.005885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f11508a2-2f1d-411c-9c31-630c4443fae0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62fdc51ecaa14ae094540a488fa74f5f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:42.102477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f11508a2-2f1d-411c-9c31-630c4443fae0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62fdc51ecaa14ae094540a488fa74f5f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:42.119067 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f11508a2-2f1d-411c-9c31-630c4443fae0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62fdc51ecaa14ae094540a488fa74f5f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:42.119902 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f11508a2-2f1d-411c-9c31-630c4443fae0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62fdc51ecaa14ae094540a488fa74f5f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:42.120534 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f11508a2-2f1d-411c-9c31-630c4443fae0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62fdc51ecaa14ae094540a488fa74f5f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:42.134583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1052/2120] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:17:41 2022] DELETE /identity/v3/users/62fdc51ecaa14ae094540a488fa74f5f => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:17:42.139504 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da24e4-8d4e-48b2-a60f-aaad732c690b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:42.140061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da24e4-8d4e-48b2-a60f-aaad732c690b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:42.140061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da24e4-8d4e-48b2-a60f-aaad732c690b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:42.160364 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e4da24e4-8d4e-48b2-a60f-aaad732c690b None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:17:42.161702 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e4da24e4-8d4e-48b2-a60f-aaad732c690b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:42.229848 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1069/2121] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:17:42.240257 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11508a2-2f1d-411c-9c31-630c4443fae0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:42.241781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f11508a2-2f1d-411c-9c31-630c4443fae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:42.304499 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11508a2-2f1d-411c-9c31-630c4443fae0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:42.308229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34ac9a6-2da1-4b9b-8144-d3e76b9c9aba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:42.308848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ac9a6-2da1-4b9b-8144-d3e76b9c9aba service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:42.309125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ac9a6-2da1-4b9b-8144-d3e76b9c9aba service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:42.309396 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ac9a6-2da1-4b9b-8144-d3e76b9c9aba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:42.319756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34ac9a6-2da1-4b9b-8144-d3e76b9c9aba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:42.321115 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34ac9a6-2da1-4b9b-8144-d3e76b9c9aba service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:42.331937 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1053/2122] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:17:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:17:42.933740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da24e4-8d4e-48b2-a60f-aaad732c690b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:42.936033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e4da24e4-8d4e-48b2-a60f-aaad732c690b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:42.950826 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da24e4-8d4e-48b2-a60f-aaad732c690b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:42.957126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07ffba-d69b-423f-af16-7a932b66fe89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:42.957863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07ffba-d69b-423f-af16-7a932b66fe89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:42.958204 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07ffba-d69b-423f-af16-7a932b66fe89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:42.958566 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07ffba-d69b-423f-af16-7a932b66fe89 admin admin] PATH_INFO: `/v3/projects/d76bce80e88b4e688ce869fd539b9d00` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:42.960549 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07ffba-d69b-423f-af16-7a932b66fe89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d76bce80e88b4e688ce869fd539b9d00)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:42.962638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07ffba-d69b-423f-af16-7a932b66fe89 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:43.085343 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ee07ffba-d69b-423f-af16-7a932b66fe89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd76bce80e88b4e688ce869fd539b9d00', 'request_id': 'req-ee07ffba-d69b-423f-af16-7a932b66fe89'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:43.091705 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ee07ffba-d69b-423f-af16-7a932b66fe89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd76bce80e88b4e688ce869fd539b9d00', 'request_id': 'req-ee07ffba-d69b-423f-af16-7a932b66fe89'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:43.237941 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ee07ffba-d69b-423f-af16-7a932b66fe89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd76bce80e88b4e688ce869fd539b9d00', 'request_id': 'req-ee07ffba-d69b-423f-af16-7a932b66fe89'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:43.250040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ee07ffba-d69b-423f-af16-7a932b66fe89 admin admin] The token cache is being invalidate because project d76bce80e88b4e688ce869fd539b9d00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:17:43.250324 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ee07ffba-d69b-423f-af16-7a932b66fe89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:43.251015 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ee07ffba-d69b-423f-af16-7a932b66fe89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:43.252580 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1070/2123] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:17:42 2022] DELETE /identity/v3/projects/d76bce80e88b4e688ce869fd539b9d00 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:17:43.257345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34ac9a6-2da1-4b9b-8144-d3e76b9c9aba service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:43.267618 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34ac9a6-2da1-4b9b-8144-d3e76b9c9aba service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:43.324918 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0901a3-908a-4250-b047-dec263512367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:43.325686 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0901a3-908a-4250-b047-dec263512367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:43.325964 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0901a3-908a-4250-b047-dec263512367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:43.326225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0901a3-908a-4250-b047-dec263512367 admin admin] PATH_INFO: `/v3/users/9ec4dfbd2b184ca4b6c1ef3f807d545d` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:43.327843 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0901a3-908a-4250-b047-dec263512367 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ec4dfbd2b184ca4b6c1ef3f807d545d)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:43.329816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0901a3-908a-4250-b047-dec263512367 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:43.348846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:17:43.611716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ea0901a3-908a-4250-b047-dec263512367 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ec4dfbd2b184ca4b6c1ef3f807d545d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:43.694454 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ea0901a3-908a-4250-b047-dec263512367 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ec4dfbd2b184ca4b6c1ef3f807d545d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:43.707849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ea0901a3-908a-4250-b047-dec263512367 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ec4dfbd2b184ca4b6c1ef3f807d545d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:43.708599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ea0901a3-908a-4250-b047-dec263512367 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ec4dfbd2b184ca4b6c1ef3f807d545d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:43.709950 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ea0901a3-908a-4250-b047-dec263512367 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ec4dfbd2b184ca4b6c1ef3f807d545d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:43.724514 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1054/2124] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:17:43 2022] DELETE /identity/v3/users/9ec4dfbd2b184ca4b6c1ef3f807d545d => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:17:44.321104 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07ffba-d69b-423f-af16-7a932b66fe89 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:44.323214 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ee07ffba-d69b-423f-af16-7a932b66fe89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:44.340888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07ffba-d69b-423f-af16-7a932b66fe89 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:44.379055 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:17:44.415463 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e5d8e-ca96-4bba-9b85-afbfa789006b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:44.416113 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e5d8e-ca96-4bba-9b85-afbfa789006b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:44.416386 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e5d8e-ca96-4bba-9b85-afbfa789006b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:44.416657 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e5d8e-ca96-4bba-9b85-afbfa789006b admin admin] PATH_INFO: `/v3/projects/1805b8e7a86148f28f5da9748b61b7d0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:44.417782 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e5d8e-ca96-4bba-9b85-afbfa789006b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1805b8e7a86148f28f5da9748b61b7d0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:44.419771 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e5d8e-ca96-4bba-9b85-afbfa789006b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:44.605359 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9b0e5d8e-ca96-4bba-9b85-afbfa789006b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1805b8e7a86148f28f5da9748b61b7d0', 'request_id': 'req-9b0e5d8e-ca96-4bba-9b85-afbfa789006b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:44.615197 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9b0e5d8e-ca96-4bba-9b85-afbfa789006b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1805b8e7a86148f28f5da9748b61b7d0', 'request_id': 'req-9b0e5d8e-ca96-4bba-9b85-afbfa789006b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:44.735765 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9b0e5d8e-ca96-4bba-9b85-afbfa789006b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1805b8e7a86148f28f5da9748b61b7d0', 'request_id': 'req-9b0e5d8e-ca96-4bba-9b85-afbfa789006b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:44.748043 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9b0e5d8e-ca96-4bba-9b85-afbfa789006b admin admin] The token cache is being invalidate because project 1805b8e7a86148f28f5da9748b61b7d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:17:44.748359 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9b0e5d8e-ca96-4bba-9b85-afbfa789006b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:44.749074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9b0e5d8e-ca96-4bba-9b85-afbfa789006b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:44.750905 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1071/2125] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:17:44 2022] DELETE /identity/v3/projects/1805b8e7a86148f28f5da9748b61b7d0 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:17:44.755868 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19395738-7b69-4857-9efd-e12ef6e061e7 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:44.757138 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19395738-7b69-4857-9efd-e12ef6e061e7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:44.757138 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19395738-7b69-4857-9efd-e12ef6e061e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:44.777580 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-19395738-7b69-4857-9efd-e12ef6e061e7 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:17:44.778842 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-19395738-7b69-4857-9efd-e12ef6e061e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:44.849053 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1055/2126] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:17:44.858280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249c8e0-129a-47a7-9262-4a1947a99177 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:44.858651 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249c8e0-129a-47a7-9262-4a1947a99177 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:44.860056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249c8e0-129a-47a7-9262-4a1947a99177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:44.883814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4249c8e0-129a-47a7-9262-4a1947a99177 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:17:44.885166 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4249c8e0-129a-47a7-9262-4a1947a99177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:44.898400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1072/2127] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:17:44.898936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19395738-7b69-4857-9efd-e12ef6e061e7 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:44.910902 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19395738-7b69-4857-9efd-e12ef6e061e7 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:44.916746 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92367cac-2053-4963-92e8-74a3c87ef21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:44.917386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92367cac-2053-4963-92e8-74a3c87ef21a admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:44.917648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92367cac-2053-4963-92e8-74a3c87ef21a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:44.917915 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92367cac-2053-4963-92e8-74a3c87ef21a admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:44.918595 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92367cac-2053-4963-92e8-74a3c87ef21a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:17:44.918892 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92367cac-2053-4963-92e8-74a3c87ef21a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:44.920495 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92367cac-2053-4963-92e8-74a3c87ef21a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:44.936735 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1056/2128] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 17:17:44.943675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4249c8e0-129a-47a7-9262-4a1947a99177 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:44.957242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4249c8e0-129a-47a7-9262-4a1947a99177 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:44.963115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc4940-cc71-4099-b11b-7469069fd6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:44.969398 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc4940-cc71-4099-b11b-7469069fd6d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:44.969398 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc4940-cc71-4099-b11b-7469069fd6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:44.969398 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc4940-cc71-4099-b11b-7469069fd6d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:44.969398 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdc4940-cc71-4099-b11b-7469069fd6d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:44.969398 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdc4940-cc71-4099-b11b-7469069fd6d5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:45.140245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1073/2129] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:17:44 2022] POST /identity/v3/projects => generated 381 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:17:45.144571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92367cac-2053-4963-92e8-74a3c87ef21a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:45.159367 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92367cac-2053-4963-92e8-74a3c87ef21a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:45.223839 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41b5d2-4e4d-4708-a8ec-fca730665660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:45.224523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41b5d2-4e4d-4708-a8ec-fca730665660 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:45.224833 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41b5d2-4e4d-4708-a8ec-fca730665660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:45.226330 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41b5d2-4e4d-4708-a8ec-fca730665660 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:45.227062 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f41b5d2-4e4d-4708-a8ec-fca730665660 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:45.228954 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f41b5d2-4e4d-4708-a8ec-fca730665660 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:45.312685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1057/2130] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:17:45 2022] POST /identity/v3/users => generated 291 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:17:45.316833 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc4940-cc71-4099-b11b-7469069fd6d5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:45.328455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc4940-cc71-4099-b11b-7469069fd6d5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:45.333178 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcce0fd-5041-4681-a735-26589d5601c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:45.333839 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcce0fd-5041-4681-a735-26589d5601c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:45.334266 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcce0fd-5041-4681-a735-26589d5601c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:45.334681 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcce0fd-5041-4681-a735-26589d5601c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:45.335367 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcce0fd-5041-4681-a735-26589d5601c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:45.337170 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcce0fd-5041-4681-a735-26589d5601c6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:45.348248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1074/2131] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:17:45.352324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41b5d2-4e4d-4708-a8ec-fca730665660 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:45.363151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41b5d2-4e4d-4708-a8ec-fca730665660 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:45.368840 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc741f6-e802-4f9d-a2fa-17d6cfe829ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:45.369447 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc741f6-e802-4f9d-a2fa-17d6cfe829ed admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:45.369721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc741f6-e802-4f9d-a2fa-17d6cfe829ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:45.369991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc741f6-e802-4f9d-a2fa-17d6cfe829ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:45.370673 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc741f6-e802-4f9d-a2fa-17d6cfe829ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:45.372700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc741f6-e802-4f9d-a2fa-17d6cfe829ed admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:45.378689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:17:45.386233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1058/2132] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:17:45.390828 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcce0fd-5041-4681-a735-26589d5601c6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:45.402723 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcce0fd-5041-4681-a735-26589d5601c6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:45.407817 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486b622-044b-45de-a181-26ed4d9be040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:45.408512 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486b622-044b-45de-a181-26ed4d9be040 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:45.409001 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486b622-044b-45de-a181-26ed4d9be040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:45.409278 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486b622-044b-45de-a181-26ed4d9be040 admin admin] PATH_INFO: `/v3/projects/7b6b5defffe24c35bec6194fff50eeea/users/65e39d8e69934d479108086b3ed7849a/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:45.428891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486b622-044b-45de-a181-26ed4d9be040 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b6b5defffe24c35bec6194fff50eeea, user_id=65e39d8e69934d479108086b3ed7849a, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:45.433687 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486b622-044b-45de-a181-26ed4d9be040 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:45.530313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1075/2133] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:17:45 2022] PUT /identity/v3/projects/7b6b5defffe24c35bec6194fff50eeea/users/65e39d8e69934d479108086b3ed7849a/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:17:45.542482 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f5ea0-9b10-4f65-9784-5bad108618ec None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:45.542896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f5ea0-9b10-4f65-9784-5bad108618ec None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:45.543267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f5ea0-9b10-4f65-9784-5bad108618ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:45.583082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-546f5ea0-9b10-4f65-9784-5bad108618ec None None] MFA Rules not processed for user `65e39d8e69934d479108086b3ed7849a`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:17:45.584408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-546f5ea0-9b10-4f65-9784-5bad108618ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:45.663773 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1059/2134] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:17:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:17:45.668418 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486b622-044b-45de-a181-26ed4d9be040 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:45.726296 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:17:45.737558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486b622-044b-45de-a181-26ed4d9be040 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:45.743187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d41f5-836d-4a0d-a760-71ca448b9939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:45.743867 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d41f5-836d-4a0d-a760-71ca448b9939 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:45.744144 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d41f5-836d-4a0d-a760-71ca448b9939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:45.744414 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d41f5-836d-4a0d-a760-71ca448b9939 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:45.745059 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d41f5-836d-4a0d-a760-71ca448b9939 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:45.747063 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d41f5-836d-4a0d-a760-71ca448b9939 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:45.839931 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1076/2135] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:17:45 2022] POST /identity/v3/projects => generated 379 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:17:45.844049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f5ea0-9b10-4f65-9784-5bad108618ec None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:45.856150 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f5ea0-9b10-4f65-9784-5bad108618ec None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:45.918748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f1c34-cb8f-480e-a8e0-0ecb4ffcf5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:45.919376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f1c34-cb8f-480e-a8e0-0ecb4ffcf5e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:45.919658 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f1c34-cb8f-480e-a8e0-0ecb4ffcf5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:45.919929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f1c34-cb8f-480e-a8e0-0ecb4ffcf5e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:45.920595 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f1c34-cb8f-480e-a8e0-0ecb4ffcf5e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:45.922563 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f1c34-cb8f-480e-a8e0-0ecb4ffcf5e0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:46.055693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1060/2136] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:17:45 2022] POST /identity/v3/users => generated 296 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:17:46.059715 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d41f5-836d-4a0d-a760-71ca448b9939 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:46.072342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d41f5-836d-4a0d-a760-71ca448b9939 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:46.080322 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf0611-88e3-4923-acfa-6373fdfedac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:46.081485 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf0611-88e3-4923-acfa-6373fdfedac1 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:46.081978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf0611-88e3-4923-acfa-6373fdfedac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:46.082498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf0611-88e3-4923-acfa-6373fdfedac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:46.083676 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf0611-88e3-4923-acfa-6373fdfedac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:46.086853 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf0611-88e3-4923-acfa-6373fdfedac1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:46.100614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1077/2137] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:17:46.105020 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f1c34-cb8f-480e-a8e0-0ecb4ffcf5e0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:46.115218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f1c34-cb8f-480e-a8e0-0ecb4ffcf5e0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:46.120441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd17273b-718a-4f06-9f08-51b30b5ee09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:46.121085 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17273b-718a-4f06-9f08-51b30b5ee09c admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:46.121357 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17273b-718a-4f06-9f08-51b30b5ee09c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:46.121641 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17273b-718a-4f06-9f08-51b30b5ee09c admin admin] PATH_INFO: `/v3/projects/2791c41d048a4381b1876299ff6ee4f6/users/7fe9e622ae6f44f0a24b323639d1db7a/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:46.137527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd17273b-718a-4f06-9f08-51b30b5ee09c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2791c41d048a4381b1876299ff6ee4f6, user_id=7fe9e622ae6f44f0a24b323639d1db7a, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:46.140225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd17273b-718a-4f06-9f08-51b30b5ee09c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:46.245133 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1061/2138] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:17:46 2022] PUT /identity/v3/projects/2791c41d048a4381b1876299ff6ee4f6/users/7fe9e622ae6f44f0a24b323639d1db7a/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:17:46.249339 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf0611-88e3-4923-acfa-6373fdfedac1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:46.260100 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf0611-88e3-4923-acfa-6373fdfedac1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:46.316234 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4f823-3164-453d-867d-df19dccfbf4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:46.316837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4f823-3164-453d-867d-df19dccfbf4b admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:46.317116 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4f823-3164-453d-867d-df19dccfbf4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:46.317390 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4f823-3164-453d-867d-df19dccfbf4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:46.318036 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de4f823-3164-453d-867d-df19dccfbf4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:46.320436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de4f823-3164-453d-867d-df19dccfbf4b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:46.330402 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1078/2139] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:17:46.334713 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd17273b-718a-4f06-9f08-51b30b5ee09c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:46.345646 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd17273b-718a-4f06-9f08-51b30b5ee09c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:46.351097 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111f615-70b9-49e5-beb9-9039f7a0005a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:46.351757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111f615-70b9-49e5-beb9-9039f7a0005a admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:46.352033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111f615-70b9-49e5-beb9-9039f7a0005a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:46.352304 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111f615-70b9-49e5-beb9-9039f7a0005a admin admin] PATH_INFO: `/v3/projects/2791c41d048a4381b1876299ff6ee4f6/users/7fe9e622ae6f44f0a24b323639d1db7a/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:46.354226 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111f615-70b9-49e5-beb9-9039f7a0005a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2791c41d048a4381b1876299ff6ee4f6, user_id=7fe9e622ae6f44f0a24b323639d1db7a, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:46.357159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111f615-70b9-49e5-beb9-9039f7a0005a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:46.368490 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1062/2140] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:17:46 2022] PUT /identity/v3/projects/2791c41d048a4381b1876299ff6ee4f6/users/7fe9e622ae6f44f0a24b323639d1db7a/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:17:46.379057 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d077f-71cf-47e9-ba7a-bf0507e20b96 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:46.379343 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d077f-71cf-47e9-ba7a-bf0507e20b96 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:46.379620 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d077f-71cf-47e9-ba7a-bf0507e20b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:46.409214 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ce1d077f-71cf-47e9-ba7a-bf0507e20b96 None None] MFA Rules not processed for user `7fe9e622ae6f44f0a24b323639d1db7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:17:46.410677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ce1d077f-71cf-47e9-ba7a-bf0507e20b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:46.483535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1079/2141] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:17:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:17:46.497106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111f615-70b9-49e5-beb9-9039f7a0005a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:46.499086 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d111f615-70b9-49e5-beb9-9039f7a0005a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:46.549690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:17:46.564517 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111f615-70b9-49e5-beb9-9039f7a0005a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:46.569022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0625d1-6c22-4854-9f82-04a7230a7f48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:46.569647 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0625d1-6c22-4854-9f82-04a7230a7f48 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:46.570012 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0625d1-6c22-4854-9f82-04a7230a7f48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:46.570302 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0625d1-6c22-4854-9f82-04a7230a7f48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:46.588868 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0625d1-6c22-4854-9f82-04a7230a7f48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:46.590442 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0625d1-6c22-4854-9f82-04a7230a7f48 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:46.601411 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1063/2142] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:17:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:17:47.212353 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d077f-71cf-47e9-ba7a-bf0507e20b96 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:47.262057 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:17:47.275653 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d077f-71cf-47e9-ba7a-bf0507e20b96 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:47.280151 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92292a39-ed9b-4132-8a17-72ed8ba5479e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:47.280768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92292a39-ed9b-4132-8a17-72ed8ba5479e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:47.281044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92292a39-ed9b-4132-8a17-72ed8ba5479e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:47.281308 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92292a39-ed9b-4132-8a17-72ed8ba5479e admin admin] PATH_INFO: `/v3/users/65e39d8e69934d479108086b3ed7849a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:47.282464 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92292a39-ed9b-4132-8a17-72ed8ba5479e admin admin] RBAC: Authorizing `identity:delete_user(user_id=65e39d8e69934d479108086b3ed7849a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:47.284388 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92292a39-ed9b-4132-8a17-72ed8ba5479e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:47.501916 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-92292a39-ed9b-4132-8a17-72ed8ba5479e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65e39d8e69934d479108086b3ed7849a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:47.603598 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-92292a39-ed9b-4132-8a17-72ed8ba5479e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65e39d8e69934d479108086b3ed7849a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:47.614838 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-92292a39-ed9b-4132-8a17-72ed8ba5479e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65e39d8e69934d479108086b3ed7849a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:47.628381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-92292a39-ed9b-4132-8a17-72ed8ba5479e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65e39d8e69934d479108086b3ed7849a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:47.629110 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-92292a39-ed9b-4132-8a17-72ed8ba5479e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65e39d8e69934d479108086b3ed7849a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:47.636315 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1080/2143] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:17:47 2022] DELETE /identity/v3/users/65e39d8e69934d479108086b3ed7849a => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:17:47.640807 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de3e3c-771d-4b70-a43f-6fb1c6b486d3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:47.641250 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de3e3c-771d-4b70-a43f-6fb1c6b486d3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:47.643725 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de3e3c-771d-4b70-a43f-6fb1c6b486d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:47.660967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-70de3e3c-771d-4b70-a43f-6fb1c6b486d3 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:17:47.662178 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-70de3e3c-771d-4b70-a43f-6fb1c6b486d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:47.723057 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1064/2144] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:17:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:17:47.735588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92292a39-ed9b-4132-8a17-72ed8ba5479e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:47.737011 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-92292a39-ed9b-4132-8a17-72ed8ba5479e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:47.803314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92292a39-ed9b-4132-8a17-72ed8ba5479e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:47.806914 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c18d2-628a-42e5-a4e9-33e2f3e91e8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:47.807521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c18d2-628a-42e5-a4e9-33e2f3e91e8a service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:47.807797 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c18d2-628a-42e5-a4e9-33e2f3e91e8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:47.808067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c18d2-628a-42e5-a4e9-33e2f3e91e8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:47.817430 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c18d2-628a-42e5-a4e9-33e2f3e91e8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:47.819565 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c18d2-628a-42e5-a4e9-33e2f3e91e8a service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:47.829596 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1081/2145] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:17:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:17:48.611911 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de3e3c-771d-4b70-a43f-6fb1c6b486d3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:48.613770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-70de3e3c-771d-4b70-a43f-6fb1c6b486d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:48.626842 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de3e3c-771d-4b70-a43f-6fb1c6b486d3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:48.630720 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd07990-d41a-41ac-a249-5cdbc78272db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:48.632346 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd07990-d41a-41ac-a249-5cdbc78272db service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:48.632622 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd07990-d41a-41ac-a249-5cdbc78272db service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:48.632888 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd07990-d41a-41ac-a249-5cdbc78272db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:48.634720 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0bd07990-d41a-41ac-a249-5cdbc78272db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:48.651762 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c18d2-628a-42e5-a4e9-33e2f3e91e8a service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:48.653413 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4a1c18d2-628a-42e5-a4e9-33e2f3e91e8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:48.668781 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c18d2-628a-42e5-a4e9-33e2f3e91e8a service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:48.674812 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909b332-80eb-45fc-801a-02b682a3e214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:48.675855 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909b332-80eb-45fc-801a-02b682a3e214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:48.676516 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909b332-80eb-45fc-801a-02b682a3e214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:48.676848 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909b332-80eb-45fc-801a-02b682a3e214 admin admin] PATH_INFO: `/v3/projects/7b6b5defffe24c35bec6194fff50eeea` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:48.678168 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909b332-80eb-45fc-801a-02b682a3e214 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b6b5defffe24c35bec6194fff50eeea)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:48.680504 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909b332-80eb-45fc-801a-02b682a3e214 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:48.711214 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd07990-d41a-41ac-a249-5cdbc78272db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:48.712693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd07990-d41a-41ac-a249-5cdbc78272db service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:48.722699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1065/2146] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:17:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:17:48.794583 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3909b332-80eb-45fc-801a-02b682a3e214 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b6b5defffe24c35bec6194fff50eeea', 'request_id': 'req-3909b332-80eb-45fc-801a-02b682a3e214'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:48.801069 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3909b332-80eb-45fc-801a-02b682a3e214 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b6b5defffe24c35bec6194fff50eeea', 'request_id': 'req-3909b332-80eb-45fc-801a-02b682a3e214'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:48.906788 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3909b332-80eb-45fc-801a-02b682a3e214 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b6b5defffe24c35bec6194fff50eeea', 'request_id': 'req-3909b332-80eb-45fc-801a-02b682a3e214'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:48.911893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:17:48.921329 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3909b332-80eb-45fc-801a-02b682a3e214 admin admin] The token cache is being invalidate because project 7b6b5defffe24c35bec6194fff50eeea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:17:48.921622 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3909b332-80eb-45fc-801a-02b682a3e214 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:48.922426 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3909b332-80eb-45fc-801a-02b682a3e214 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:48.923954 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1082/2147] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:17:48 2022] DELETE /identity/v3/projects/7b6b5defffe24c35bec6194fff50eeea => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:17:48.928773 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd07990-d41a-41ac-a249-5cdbc78272db service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:48.944331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd07990-d41a-41ac-a249-5cdbc78272db service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:49.024660 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e82044-3468-46f8-85f1-4cf63dc7c16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:49.025357 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e82044-3468-46f8-85f1-4cf63dc7c16a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:49.025634 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e82044-3468-46f8-85f1-4cf63dc7c16a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:49.025902 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e82044-3468-46f8-85f1-4cf63dc7c16a admin admin] PATH_INFO: `/v3/users/7fe9e622ae6f44f0a24b323639d1db7a` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:49.027223 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e82044-3468-46f8-85f1-4cf63dc7c16a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fe9e622ae6f44f0a24b323639d1db7a)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:49.029262 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e82044-3468-46f8-85f1-4cf63dc7c16a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:49.063169 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:17:49.247203 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-53e82044-3468-46f8-85f1-4cf63dc7c16a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fe9e622ae6f44f0a24b323639d1db7a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:49.358132 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-53e82044-3468-46f8-85f1-4cf63dc7c16a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fe9e622ae6f44f0a24b323639d1db7a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:49.374695 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-53e82044-3468-46f8-85f1-4cf63dc7c16a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fe9e622ae6f44f0a24b323639d1db7a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:49.375546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-53e82044-3468-46f8-85f1-4cf63dc7c16a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fe9e622ae6f44f0a24b323639d1db7a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:49.376271 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-53e82044-3468-46f8-85f1-4cf63dc7c16a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fe9e622ae6f44f0a24b323639d1db7a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:49.391692 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1066/2148] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:17:48 2022] DELETE /identity/v3/users/7fe9e622ae6f44f0a24b323639d1db7a => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:17:49.899013 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909b332-80eb-45fc-801a-02b682a3e214 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:49.901307 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3909b332-80eb-45fc-801a-02b682a3e214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:49.930896 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909b332-80eb-45fc-801a-02b682a3e214 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:50.003371 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b8e5c-1cb4-461a-995b-8bff9e680345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:50.004066 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b8e5c-1cb4-461a-995b-8bff9e680345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:50.004267 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b8e5c-1cb4-461a-995b-8bff9e680345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:50.004541 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b8e5c-1cb4-461a-995b-8bff9e680345 admin admin] PATH_INFO: `/v3/projects/2791c41d048a4381b1876299ff6ee4f6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:50.005714 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b8e5c-1cb4-461a-995b-8bff9e680345 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2791c41d048a4381b1876299ff6ee4f6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:50.008101 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b8e5c-1cb4-461a-995b-8bff9e680345 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:50.165407 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-259b8e5c-1cb4-461a-995b-8bff9e680345 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2791c41d048a4381b1876299ff6ee4f6', 'request_id': 'req-259b8e5c-1cb4-461a-995b-8bff9e680345'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:50.171898 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-259b8e5c-1cb4-461a-995b-8bff9e680345 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2791c41d048a4381b1876299ff6ee4f6', 'request_id': 'req-259b8e5c-1cb4-461a-995b-8bff9e680345'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:50.239009 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-259b8e5c-1cb4-461a-995b-8bff9e680345 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2791c41d048a4381b1876299ff6ee4f6', 'request_id': 'req-259b8e5c-1cb4-461a-995b-8bff9e680345'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:50.252558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-259b8e5c-1cb4-461a-995b-8bff9e680345 admin admin] The token cache is being invalidate because project 2791c41d048a4381b1876299ff6ee4f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:17:50.252841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-259b8e5c-1cb4-461a-995b-8bff9e680345 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:50.253550 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-259b8e5c-1cb4-461a-995b-8bff9e680345 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:17:50.255132 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1083/2149] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:17:49 2022] DELETE /identity/v3/projects/2791c41d048a4381b1876299ff6ee4f6 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:17:50.260943 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc14bcb-b178-4560-b58f-05aa8ef3ea50 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:50.261490 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc14bcb-b178-4560-b58f-05aa8ef3ea50 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:50.261490 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc14bcb-b178-4560-b58f-05aa8ef3ea50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:50.282416 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3bc14bcb-b178-4560-b58f-05aa8ef3ea50 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:17:50.283689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3bc14bcb-b178-4560-b58f-05aa8ef3ea50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:50.354440 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1067/2150] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:17:50.363357 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c81ea-1976-43c7-9549-3c45d04e20e4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:50.363640 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c81ea-1976-43c7-9549-3c45d04e20e4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:50.363911 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c81ea-1976-43c7-9549-3c45d04e20e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:50.386643 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-652c81ea-1976-43c7-9549-3c45d04e20e4 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:17:50.387841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-652c81ea-1976-43c7-9549-3c45d04e20e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:50.396186 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1084/2151] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:17:50.401410 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc14bcb-b178-4560-b58f-05aa8ef3ea50 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:50.411426 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc14bcb-b178-4560-b58f-05aa8ef3ea50 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:50.417400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc16bc-afcf-4776-bc23-9e8406cb42ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:50.417999 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc16bc-afcf-4776-bc23-9e8406cb42ec admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:50.418276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc16bc-afcf-4776-bc23-9e8406cb42ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:50.418586 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc16bc-afcf-4776-bc23-9e8406cb42ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:50.419713 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc16bc-afcf-4776-bc23-9e8406cb42ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:17:50.420419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc16bc-afcf-4776-bc23-9e8406cb42ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:50.423564 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc16bc-afcf-4776-bc23-9e8406cb42ec admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:50.439626 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1068/2152] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 17:17:50.444578 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c81ea-1976-43c7-9549-3c45d04e20e4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:50.455166 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c81ea-1976-43c7-9549-3c45d04e20e4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:50.461871 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b3fe9-b2d4-4fc1-bdd6-0c7a7f249e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:50.462519 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b3fe9-b2d4-4fc1-bdd6-0c7a7f249e67 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:50.462802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b3fe9-b2d4-4fc1-bdd6-0c7a7f249e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:50.463074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b3fe9-b2d4-4fc1-bdd6-0c7a7f249e67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:50.463716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b3fe9-b2d4-4fc1-bdd6-0c7a7f249e67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:50.465778 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b3fe9-b2d4-4fc1-bdd6-0c7a7f249e67 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:50.561064 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1085/2153] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:17:50 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:17:50.565719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc16bc-afcf-4776-bc23-9e8406cb42ec admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:50.578068 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc16bc-afcf-4776-bc23-9e8406cb42ec admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:50.648943 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196fcfc7-4681-47f9-8d20-131abccf0b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:50.649565 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fcfc7-4681-47f9-8d20-131abccf0b56 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:50.651515 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fcfc7-4681-47f9-8d20-131abccf0b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:50.651515 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fcfc7-4681-47f9-8d20-131abccf0b56 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:50.651515 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196fcfc7-4681-47f9-8d20-131abccf0b56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:50.652833 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196fcfc7-4681-47f9-8d20-131abccf0b56 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:50.799837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1069/2154] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:17:50 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:17:50.802217 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b3fe9-b2d4-4fc1-bdd6-0c7a7f249e67 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:50.813448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b3fe9-b2d4-4fc1-bdd6-0c7a7f249e67 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:50.818205 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866c2f6-f787-4ddc-850b-a45b7428dada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:50.819487 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866c2f6-f787-4ddc-850b-a45b7428dada admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:50.819748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866c2f6-f787-4ddc-850b-a45b7428dada admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:50.820019 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866c2f6-f787-4ddc-850b-a45b7428dada admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:50.820672 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866c2f6-f787-4ddc-850b-a45b7428dada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:50.822377 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866c2f6-f787-4ddc-850b-a45b7428dada admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:50.833318 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1086/2155] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:17:50.837469 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196fcfc7-4681-47f9-8d20-131abccf0b56 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:50.847862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196fcfc7-4681-47f9-8d20-131abccf0b56 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:50.853270 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e3106-bbce-43ef-8379-604a92738722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:50.853879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e3106-bbce-43ef-8379-604a92738722 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:50.854296 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e3106-bbce-43ef-8379-604a92738722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:50.854603 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e3106-bbce-43ef-8379-604a92738722 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:50.855253 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e3106-bbce-43ef-8379-604a92738722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:50.859834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e3106-bbce-43ef-8379-604a92738722 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:50.869146 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1070/2156] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:17:50.873554 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866c2f6-f787-4ddc-850b-a45b7428dada admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:50.897550 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866c2f6-f787-4ddc-850b-a45b7428dada admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:50.897550 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eedbf4a-de9e-4ffe-a71f-c43c8b938058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:50.897550 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedbf4a-de9e-4ffe-a71f-c43c8b938058 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:50.897550 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedbf4a-de9e-4ffe-a71f-c43c8b938058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:50.898022 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedbf4a-de9e-4ffe-a71f-c43c8b938058 admin admin] PATH_INFO: `/v3/projects/e4c58771ede14b0fae72c65ade80094d/users/1bdecf3dbbe14097957966359aa62c6d/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:50.920310 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eedbf4a-de9e-4ffe-a71f-c43c8b938058 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4c58771ede14b0fae72c65ade80094d, user_id=1bdecf3dbbe14097957966359aa62c6d, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:50.923648 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eedbf4a-de9e-4ffe-a71f-c43c8b938058 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:51.033961 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1087/2157] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:17:50 2022] PUT /identity/v3/projects/e4c58771ede14b0fae72c65ade80094d/users/1bdecf3dbbe14097957966359aa62c6d/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:17:51.048685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab9ae7-5b51-4244-9c42-4ba2d44023fd None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:51.049123 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab9ae7-5b51-4244-9c42-4ba2d44023fd None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:51.049402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab9ae7-5b51-4244-9c42-4ba2d44023fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:51.064854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:17:51.095411 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f4ab9ae7-5b51-4244-9c42-4ba2d44023fd None None] MFA Rules not processed for user `1bdecf3dbbe14097957966359aa62c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:17:51.096810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f4ab9ae7-5b51-4244-9c42-4ba2d44023fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:51.196361 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1071/2158] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:17:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:17:51.212904 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eedbf4a-de9e-4ffe-a71f-c43c8b938058 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:51.218201 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6eedbf4a-de9e-4ffe-a71f-c43c8b938058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:51.245675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:17:51.292602 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eedbf4a-de9e-4ffe-a71f-c43c8b938058 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:51.296792 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde06468-6d88-44d6-9fa2-5e4942effc43 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10f4f54890624da29ff217081445b6bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:51.297589 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde06468-6d88-44d6-9fa2-5e4942effc43 service glance] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:51.297871 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde06468-6d88-44d6-9fa2-5e4942effc43 service glance] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:51.298144 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde06468-6d88-44d6-9fa2-5e4942effc43 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:51.308691 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde06468-6d88-44d6-9fa2-5e4942effc43 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:51.310621 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde06468-6d88-44d6-9fa2-5e4942effc43 service glance] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:51.322060 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1088/2159] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:17:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:17:51.837040 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64eb75-8490-4ecf-b384-e61b12d5d22d None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:51.837615 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64eb75-8490-4ecf-b384-e61b12d5d22d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:51.837615 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64eb75-8490-4ecf-b384-e61b12d5d22d None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:51.839338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1072/2160] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 10 17:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:17:51.848866 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87be85-4632-42c2-a182-7eed6f09e7ac None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:51.849206 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87be85-4632-42c2-a182-7eed6f09e7ac None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:51.849633 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87be85-4632-42c2-a182-7eed6f09e7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:51.883650 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ad87be85-4632-42c2-a182-7eed6f09e7ac None None] MFA Rules not processed for user `615cc36e21cc493b823e6cee695ac308`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:17:51.884896 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ad87be85-4632-42c2-a182-7eed6f09e7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:51.985618 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1089/2161] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 10 17:17:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:17:52.017665 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64eb75-8490-4ecf-b384-e61b12d5d22d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:52.028341 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64eb75-8490-4ecf-b384-e61b12d5d22d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:52.032487 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f56b43-e21c-46f3-82af-f3eed8f661ad service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10f4f54890624da29ff217081445b6bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:52.033094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f56b43-e21c-46f3-82af-f3eed8f661ad service glance] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:52.033368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f56b43-e21c-46f3-82af-f3eed8f661ad service glance] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:52.033634 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f56b43-e21c-46f3-82af-f3eed8f661ad service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:52.049091 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f56b43-e21c-46f3-82af-f3eed8f661ad service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:52.050768 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f56b43-e21c-46f3-82af-f3eed8f661ad service glance] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:52.075182 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1073/2162] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:17:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:17:52.369341 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfc0bd-8558-414d-a2d7-9025af265eed None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:52.369629 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfc0bd-8558-414d-a2d7-9025af265eed None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:52.369902 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfc0bd-8558-414d-a2d7-9025af265eed None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:52.371531 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1090/2163] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 10 17:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:17:52.377100 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd75aca-4be1-4a5a-b43f-4e8451530c5f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:52.377361 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd75aca-4be1-4a5a-b43f-4e8451530c5f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:52.377634 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd75aca-4be1-4a5a-b43f-4e8451530c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:52.398939 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-bbd75aca-4be1-4a5a-b43f-4e8451530c5f None None] MFA Rules not processed for user `e1534cfef9e9428bb2e010fff200bc19`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:17:52.400195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bbd75aca-4be1-4a5a-b43f-4e8451530c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:52.466998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1074/2164] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 10 17:17:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:17:52.472467 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bfc0bd-8558-414d-a2d7-9025af265eed None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:52.474212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-38bfc0bd-8558-414d-a2d7-9025af265eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:52.495809 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bfc0bd-8558-414d-a2d7-9025af265eed None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:52.499935 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ecc57-026f-48fc-b073-eaa1d450373c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:52.500551 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ecc57-026f-48fc-b073-eaa1d450373c service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:52.500830 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ecc57-026f-48fc-b073-eaa1d450373c service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:52.501104 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ecc57-026f-48fc-b073-eaa1d450373c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:52.509180 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ecc57-026f-48fc-b073-eaa1d450373c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:52.511121 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ecc57-026f-48fc-b073-eaa1d450373c service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:52.520624 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1091/2165] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:17:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:17:52.624691 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239b97b-d566-4eb7-9503-471b57b29d7d None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:52.624979 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239b97b-d566-4eb7-9503-471b57b29d7d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:52.625295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239b97b-d566-4eb7-9503-471b57b29d7d None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:52.627082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1075/2166] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 10 17:17:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:17:52.634000 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbc268-ea70-4f13-9bb1-48ad4bc76535 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:52.634287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbc268-ea70-4f13-9bb1-48ad4bc76535 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:52.634598 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbc268-ea70-4f13-9bb1-48ad4bc76535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:52.661878 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c3dbc268-ea70-4f13-9bb1-48ad4bc76535 None None] MFA Rules not processed for user `615cc36e21cc493b823e6cee695ac308`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:17:52.663944 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c3dbc268-ea70-4f13-9bb1-48ad4bc76535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:52.686416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1092/2167] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 10 17:17:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:17:52.699212 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239b97b-d566-4eb7-9503-471b57b29d7d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:52.700632 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a239b97b-d566-4eb7-9503-471b57b29d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:52.715027 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239b97b-d566-4eb7-9503-471b57b29d7d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:52.718688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ee2a3-506f-4b0c-9bb7-66242e9f9ea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:52.721946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ee2a3-506f-4b0c-9bb7-66242e9f9ea4 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:52.721946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ee2a3-506f-4b0c-9bb7-66242e9f9ea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:52.721946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ee2a3-506f-4b0c-9bb7-66242e9f9ea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:52.731245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390ee2a3-506f-4b0c-9bb7-66242e9f9ea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:52.732681 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390ee2a3-506f-4b0c-9bb7-66242e9f9ea4 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:52.744857 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1076/2168] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:17:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:17:57.265029 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392f80c-6975-4d80-95c1-fee7309af7d3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:57.265557 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392f80c-6975-4d80-95c1-fee7309af7d3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:57.265731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392f80c-6975-4d80-95c1-fee7309af7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:57.285683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f392f80c-6975-4d80-95c1-fee7309af7d3 None None] MFA Rules not processed for user `4f9e1767ec90435f80248a2930081265`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:17:57.287699 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f392f80c-6975-4d80-95c1-fee7309af7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:57.347613 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1093/2169] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 10 17:17:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:17:57.361545 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ee2a3-506f-4b0c-9bb7-66242e9f9ea4 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:17:57.363332 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-390ee2a3-506f-4b0c-9bb7-66242e9f9ea4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:17:57.454706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ee2a3-506f-4b0c-9bb7-66242e9f9ea4 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:17:57.461079 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184465b9-ef96-4633-a198-a5ebf279646f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4986821ef9f84b99a835e387635aa9a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:17:57.461710 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184465b9-ef96-4633-a198-a5ebf279646f service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:17:57.461982 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184465b9-ef96-4633-a198-a5ebf279646f service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:17:57.462249 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184465b9-ef96-4633-a198-a5ebf279646f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:17:57.473387 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184465b9-ef96-4633-a198-a5ebf279646f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:17:57.475688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184465b9-ef96-4633-a198-a5ebf279646f service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:17:57.484440 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1077/2170] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 10 17:17:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 17:18:07.339325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=82 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:18:07.481054 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:18:13.666882 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef9d7f-eb27-439f-b197-94038ee93a21 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:13.667705 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef9d7f-eb27-439f-b197-94038ee93a21 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:13.667705 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef9d7f-eb27-439f-b197-94038ee93a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:13.707442 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2bef9d7f-eb27-439f-b197-94038ee93a21 None None] MFA Rules not processed for user `4f9e1767ec90435f80248a2930081265`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:18:13.709770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2bef9d7f-eb27-439f-b197-94038ee93a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:13.721733 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1094/2171] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 10 17:18:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:18:13.743666 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184465b9-ef96-4633-a198-a5ebf279646f service swift] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:13.776513 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184465b9-ef96-4633-a198-a5ebf279646f service swift] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:13.787172 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88313d5-bd44-473a-a9d6-2299e5fb6f71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4986821ef9f84b99a835e387635aa9a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:13.788259 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88313d5-bd44-473a-a9d6-2299e5fb6f71 service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:13.788759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88313d5-bd44-473a-a9d6-2299e5fb6f71 service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:13.789578 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88313d5-bd44-473a-a9d6-2299e5fb6f71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:13.806835 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88313d5-bd44-473a-a9d6-2299e5fb6f71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:13.810086 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88313d5-bd44-473a-a9d6-2299e5fb6f71 service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:13.832043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1078/2172] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 10 17:18:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 17:18:23.661899 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3dd46a-6350-448c-be57-7d8980ddeb3c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:23.667633 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3dd46a-6350-448c-be57-7d8980ddeb3c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:23.667633 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3dd46a-6350-448c-be57-7d8980ddeb3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:23.682332 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-fd3dd46a-6350-448c-be57-7d8980ddeb3c None None] MFA Rules not processed for user `4f9e1767ec90435f80248a2930081265`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:18:23.683572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fd3dd46a-6350-448c-be57-7d8980ddeb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:23.691217 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1095/2173] 10.222.0.45 () {58 vars in 967 bytes} [Wed Aug 10 17:18:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:18:23.705276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88313d5-bd44-473a-a9d6-2299e5fb6f71 service swift] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:23.718406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88313d5-bd44-473a-a9d6-2299e5fb6f71 service swift] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:23.729592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51215d-49ff-4594-8691-e719f4eebcf1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4986821ef9f84b99a835e387635aa9a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:23.730304 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51215d-49ff-4594-8691-e719f4eebcf1 service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:23.730683 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51215d-49ff-4594-8691-e719f4eebcf1 service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:23.731229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51215d-49ff-4594-8691-e719f4eebcf1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:23.754681 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51215d-49ff-4594-8691-e719f4eebcf1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:23.756444 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51215d-49ff-4594-8691-e719f4eebcf1 service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:23.767603 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1079/2174] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 10 17:18:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 17:18:27.701485 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b13ec4-67b2-47bf-8a6e-9ab6f88f3da8 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:27.702063 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b13ec4-67b2-47bf-8a6e-9ab6f88f3da8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:27.702063 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b13ec4-67b2-47bf-8a6e-9ab6f88f3da8 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:27.703511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1096/2175] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 10 17:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:18:27.747561 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928df4f-d1b0-47a3-b410-8d545b3dee64 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:27.748120 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928df4f-d1b0-47a3-b410-8d545b3dee64 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:27.748120 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928df4f-d1b0-47a3-b410-8d545b3dee64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:27.768292 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f928df4f-d1b0-47a3-b410-8d545b3dee64 None None] MFA Rules not processed for user `8cd39d9a583a4d73a5654017be106019`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:18:27.769673 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f928df4f-d1b0-47a3-b410-8d545b3dee64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:27.840030 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1080/2176] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 10 17:18:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:18:28.029195 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b13ec4-67b2-47bf-8a6e-9ab6f88f3da8 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:28.031866 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f2b13ec4-67b2-47bf-8a6e-9ab6f88f3da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:28.060757 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b13ec4-67b2-47bf-8a6e-9ab6f88f3da8 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:28.065805 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce897342-a781-42e7-a71b-3fa1641643d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cd39d9a583a4d73a5654017be106019', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:28.066438 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce897342-a781-42e7-a71b-3fa1641643d3 service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:28.066716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce897342-a781-42e7-a71b-3fa1641643d3 service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:28.066987 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce897342-a781-42e7-a71b-3fa1641643d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:28.076295 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce897342-a781-42e7-a71b-3fa1641643d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:28.077877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce897342-a781-42e7-a71b-3fa1641643d3 service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:28.089565 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1097/2177] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:18:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:18:28.195442 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928df4f-d1b0-47a3-b410-8d545b3dee64 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:28.197584 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f928df4f-d1b0-47a3-b410-8d545b3dee64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:28.293311 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928df4f-d1b0-47a3-b410-8d545b3dee64 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:28.298806 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51903f-60f2-4290-9afe-ec1b033f8d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:28.299733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51903f-60f2-4290-9afe-ec1b033f8d18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:28.300194 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51903f-60f2-4290-9afe-ec1b033f8d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:28.300708 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51903f-60f2-4290-9afe-ec1b033f8d18 admin admin] PATH_INFO: `/v3/users/44132cbd18f2470dbe9de6b5125d7db5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:28.302928 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51903f-60f2-4290-9afe-ec1b033f8d18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44132cbd18f2470dbe9de6b5125d7db5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:28.305825 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51903f-60f2-4290-9afe-ec1b033f8d18 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:28.578855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4b51903f-60f2-4290-9afe-ec1b033f8d18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44132cbd18f2470dbe9de6b5125d7db5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:28.706507 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4b51903f-60f2-4290-9afe-ec1b033f8d18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44132cbd18f2470dbe9de6b5125d7db5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:28.724006 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4b51903f-60f2-4290-9afe-ec1b033f8d18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44132cbd18f2470dbe9de6b5125d7db5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:28.725603 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4b51903f-60f2-4290-9afe-ec1b033f8d18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44132cbd18f2470dbe9de6b5125d7db5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:28.726224 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4b51903f-60f2-4290-9afe-ec1b033f8d18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44132cbd18f2470dbe9de6b5125d7db5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:28.730091 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:18:28.743598 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1081/2178] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:18:28 2022] DELETE /identity/v3/users/44132cbd18f2470dbe9de6b5125d7db5 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:18:29.097928 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce897342-a781-42e7-a71b-3fa1641643d3 service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:29.124186 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce897342-a781-42e7-a71b-3fa1641643d3 service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:29.232790 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745a639-a796-499e-83dc-e9c819aabcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:29.233524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745a639-a796-499e-83dc-e9c819aabcf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:29.233902 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745a639-a796-499e-83dc-e9c819aabcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:29.234720 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745a639-a796-499e-83dc-e9c819aabcf3 admin admin] PATH_INFO: `/v3/projects/f827cc75b1c449f6a82d5c23af901fa0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:29.235670 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745a639-a796-499e-83dc-e9c819aabcf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f827cc75b1c449f6a82d5c23af901fa0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:29.237782 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745a639-a796-499e-83dc-e9c819aabcf3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:29.388958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6745a639-a796-499e-83dc-e9c819aabcf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f827cc75b1c449f6a82d5c23af901fa0', 'request_id': 'req-6745a639-a796-499e-83dc-e9c819aabcf3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:29.398776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6745a639-a796-499e-83dc-e9c819aabcf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f827cc75b1c449f6a82d5c23af901fa0', 'request_id': 'req-6745a639-a796-499e-83dc-e9c819aabcf3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:29.471739 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6745a639-a796-499e-83dc-e9c819aabcf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f827cc75b1c449f6a82d5c23af901fa0', 'request_id': 'req-6745a639-a796-499e-83dc-e9c819aabcf3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:29.488245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6745a639-a796-499e-83dc-e9c819aabcf3 admin admin] The token cache is being invalidate because project f827cc75b1c449f6a82d5c23af901fa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:18:29.489107 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6745a639-a796-499e-83dc-e9c819aabcf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:29.489525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6745a639-a796-499e-83dc-e9c819aabcf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:29.492356 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1098/2179] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:18:29 2022] DELETE /identity/v3/projects/f827cc75b1c449f6a82d5c23af901fa0 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:18:29.497106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8348a-f234-43fe-9420-93a0526d59a0 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:29.497609 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8348a-f234-43fe-9420-93a0526d59a0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:29.497798 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8348a-f234-43fe-9420-93a0526d59a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:29.528893 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-dfb8348a-f234-43fe-9420-93a0526d59a0 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:18:29.530114 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dfb8348a-f234-43fe-9420-93a0526d59a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:29.597171 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1082/2180] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:18:29.604956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611dd2b-b297-427a-9ba6-91c0be0fb05d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:29.605243 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611dd2b-b297-427a-9ba6-91c0be0fb05d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:29.605540 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611dd2b-b297-427a-9ba6-91c0be0fb05d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:29.633566 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7611dd2b-b297-427a-9ba6-91c0be0fb05d None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:18:29.635238 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7611dd2b-b297-427a-9ba6-91c0be0fb05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:29.644441 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1099/2181] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:18:29.649730 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8348a-f234-43fe-9420-93a0526d59a0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:29.659471 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8348a-f234-43fe-9420-93a0526d59a0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:29.664654 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb71da-697c-42e6-a320-5f2e7ccfa0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:29.665507 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb71da-697c-42e6-a320-5f2e7ccfa0ce admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:29.666069 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb71da-697c-42e6-a320-5f2e7ccfa0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:29.666847 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb71da-697c-42e6-a320-5f2e7ccfa0ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:29.667567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb71da-697c-42e6-a320-5f2e7ccfa0ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:18:29.667911 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb71da-697c-42e6-a320-5f2e7ccfa0ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:29.669886 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb71da-697c-42e6-a320-5f2e7ccfa0ce admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:29.682737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1083/2182] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:18:29.687171 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611dd2b-b297-427a-9ba6-91c0be0fb05d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:29.697728 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611dd2b-b297-427a-9ba6-91c0be0fb05d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:29.702629 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a873ec-1142-4036-acd9-60816ac19a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:29.703297 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a873ec-1142-4036-acd9-60816ac19a6b admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:29.703571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a873ec-1142-4036-acd9-60816ac19a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:29.703873 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a873ec-1142-4036-acd9-60816ac19a6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:29.704485 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a873ec-1142-4036-acd9-60816ac19a6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:29.706467 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a873ec-1142-4036-acd9-60816ac19a6b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:29.773690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb71da-697c-42e6-a320-5f2e7ccfa0ce admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:29.775788 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-64eb71da-697c-42e6-a320-5f2e7ccfa0ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:29.792089 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb71da-697c-42e6-a320-5f2e7ccfa0ce admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:29.800197 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fac4dd-0435-4908-b710-14d299eab471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:29.800846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fac4dd-0435-4908-b710-14d299eab471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:29.801123 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fac4dd-0435-4908-b710-14d299eab471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:29.801397 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fac4dd-0435-4908-b710-14d299eab471 admin admin] PATH_INFO: `/v3/users/1bdecf3dbbe14097957966359aa62c6d` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:29.803692 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fac4dd-0435-4908-b710-14d299eab471 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bdecf3dbbe14097957966359aa62c6d)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:29.805690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fac4dd-0435-4908-b710-14d299eab471 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:29.806248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1100/2183] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:18:29 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:18:29.811958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a873ec-1142-4036-acd9-60816ac19a6b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:29.858240 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:18:29.882047 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a873ec-1142-4036-acd9-60816ac19a6b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:29.887475 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fdc13b-051c-4f3d-8226-2e468e46a260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:29.888082 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdc13b-051c-4f3d-8226-2e468e46a260 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:29.888360 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdc13b-051c-4f3d-8226-2e468e46a260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:29.888760 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdc13b-051c-4f3d-8226-2e468e46a260 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:29.889389 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fdc13b-051c-4f3d-8226-2e468e46a260 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:29.891448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fdc13b-051c-4f3d-8226-2e468e46a260 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:30.050181 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1101/2184] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:18:29 2022] POST /identity/v3/users => generated 287 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:18:30.054801 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fdc13b-051c-4f3d-8226-2e468e46a260 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:30.066289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fdc13b-051c-4f3d-8226-2e468e46a260 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:30.082199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5acf9-558c-4e82-b4b1-e6ca6792301e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:30.083391 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5acf9-558c-4e82-b4b1-e6ca6792301e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:30.085024 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5acf9-558c-4e82-b4b1-e6ca6792301e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:30.085024 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5acf9-558c-4e82-b4b1-e6ca6792301e admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:30.085024 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f5acf9-558c-4e82-b4b1-e6ca6792301e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:30.086747 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f5acf9-558c-4e82-b4b1-e6ca6792301e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:30.093236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d7fac4dd-0435-4908-b710-14d299eab471 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bdecf3dbbe14097957966359aa62c6d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:30.112584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1102/2185] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:18:30.114395 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5acf9-558c-4e82-b4b1-e6ca6792301e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:30.127171 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5acf9-558c-4e82-b4b1-e6ca6792301e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:30.132924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab566c0-e62c-4bff-ae62-2fdde2b8338f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:30.133549 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab566c0-e62c-4bff-ae62-2fdde2b8338f admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:30.133835 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab566c0-e62c-4bff-ae62-2fdde2b8338f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:30.134108 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab566c0-e62c-4bff-ae62-2fdde2b8338f admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:30.134802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab566c0-e62c-4bff-ae62-2fdde2b8338f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:30.136677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab566c0-e62c-4bff-ae62-2fdde2b8338f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:30.154570 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1103/2186] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:18:30.156781 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab566c0-e62c-4bff-ae62-2fdde2b8338f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:30.168491 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d7fac4dd-0435-4908-b710-14d299eab471 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bdecf3dbbe14097957966359aa62c6d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:30.169192 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab566c0-e62c-4bff-ae62-2fdde2b8338f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:30.175059 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477dc89-dfea-411f-a2ad-796799737fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:30.175831 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477dc89-dfea-411f-a2ad-796799737fd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:30.176146 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477dc89-dfea-411f-a2ad-796799737fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:30.176419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477dc89-dfea-411f-a2ad-796799737fd0 admin admin] PATH_INFO: `/v3/projects/4a0e1b0bf3334765bec6efaa5b2ba49a/users/5ad9e1bb2c9240e4b94d4313d9ab4bbe/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:30.193411 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d7fac4dd-0435-4908-b710-14d299eab471 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bdecf3dbbe14097957966359aa62c6d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:30.195734 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0477dc89-dfea-411f-a2ad-796799737fd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a0e1b0bf3334765bec6efaa5b2ba49a, user_id=5ad9e1bb2c9240e4b94d4313d9ab4bbe, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:30.198427 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0477dc89-dfea-411f-a2ad-796799737fd0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:30.198874 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d7fac4dd-0435-4908-b710-14d299eab471 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bdecf3dbbe14097957966359aa62c6d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:30.200137 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d7fac4dd-0435-4908-b710-14d299eab471 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bdecf3dbbe14097957966359aa62c6d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:30.227700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1084/2187] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:18:29 2022] DELETE /identity/v3/users/1bdecf3dbbe14097957966359aa62c6d => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:18:30.232066 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de15d14-8c4e-444a-9ffa-acd07a00fdc4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:30.232346 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de15d14-8c4e-444a-9ffa-acd07a00fdc4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:30.236027 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de15d14-8c4e-444a-9ffa-acd07a00fdc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:30.264993 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1de15d14-8c4e-444a-9ffa-acd07a00fdc4 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:18:30.265295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1de15d14-8c4e-444a-9ffa-acd07a00fdc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:30.299355 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1104/2188] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:18:30 2022] PUT /identity/v3/projects/4a0e1b0bf3334765bec6efaa5b2ba49a/users/5ad9e1bb2c9240e4b94d4313d9ab4bbe/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:18:30.310383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ca225-ea73-48ca-8de0-ee61760c0061 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:30.310751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ca225-ea73-48ca-8de0-ee61760c0061 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:30.311105 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ca225-ea73-48ca-8de0-ee61760c0061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:30.353044 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:18:30.364717 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3a7ca225-ea73-48ca-8de0-ee61760c0061 None None] MFA Rules not processed for user `5ad9e1bb2c9240e4b94d4313d9ab4bbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:18:30.366119 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3a7ca225-ea73-48ca-8de0-ee61760c0061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:30.402827 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1085/2189] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:18:30.418413 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de15d14-8c4e-444a-9ffa-acd07a00fdc4 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:30.421325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1de15d14-8c4e-444a-9ffa-acd07a00fdc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:30.477704 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1105/2190] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:18:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:18:30.489975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ca225-ea73-48ca-8de0-ee61760c0061 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:30.491136 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3a7ca225-ea73-48ca-8de0-ee61760c0061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:30.514072 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de15d14-8c4e-444a-9ffa-acd07a00fdc4 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:30.560628 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ca225-ea73-48ca-8de0-ee61760c0061 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:30.569753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85fcc2f-2a27-40e7-b6dc-9d4a6e43bfaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:30.570594 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fcc2f-2a27-40e7-b6dc-9d4a6e43bfaf service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:30.570880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fcc2f-2a27-40e7-b6dc-9d4a6e43bfaf service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:30.571152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fcc2f-2a27-40e7-b6dc-9d4a6e43bfaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:30.581846 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85fcc2f-2a27-40e7-b6dc-9d4a6e43bfaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:30.584537 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70162ec0-3d7c-4ed9-be35-1a77d3656498 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:30.584709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85fcc2f-2a27-40e7-b6dc-9d4a6e43bfaf service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:30.585514 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70162ec0-3d7c-4ed9-be35-1a77d3656498 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:30.585800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70162ec0-3d7c-4ed9-be35-1a77d3656498 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:30.586073 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70162ec0-3d7c-4ed9-be35-1a77d3656498 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:30.595285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1106/2191] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:18:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:18:30.636168 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85fcc2f-2a27-40e7-b6dc-9d4a6e43bfaf service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:30.637713 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b85fcc2f-2a27-40e7-b6dc-9d4a6e43bfaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:30.657240 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:18:30.672269 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70162ec0-3d7c-4ed9-be35-1a77d3656498 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:30.673910 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70162ec0-3d7c-4ed9-be35-1a77d3656498 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:30.686837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1086/2192] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:18:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:18:30.726905 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85fcc2f-2a27-40e7-b6dc-9d4a6e43bfaf service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:30.732438 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b0fd5-2ea6-4138-af01-a243365b6fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:30.733212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b0fd5-2ea6-4138-af01-a243365b6fc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:30.733498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b0fd5-2ea6-4138-af01-a243365b6fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:30.733769 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b0fd5-2ea6-4138-af01-a243365b6fc6 admin admin] PATH_INFO: `/v3/users/5ad9e1bb2c9240e4b94d4313d9ab4bbe` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:30.734948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b0fd5-2ea6-4138-af01-a243365b6fc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ad9e1bb2c9240e4b94d4313d9ab4bbe)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:30.737061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b0fd5-2ea6-4138-af01-a243365b6fc6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:31.055802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-315b0fd5-2ea6-4138-af01-a243365b6fc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ad9e1bb2c9240e4b94d4313d9ab4bbe'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:31.162561 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-315b0fd5-2ea6-4138-af01-a243365b6fc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ad9e1bb2c9240e4b94d4313d9ab4bbe'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:31.173031 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-315b0fd5-2ea6-4138-af01-a243365b6fc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ad9e1bb2c9240e4b94d4313d9ab4bbe'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:31.190007 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-315b0fd5-2ea6-4138-af01-a243365b6fc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ad9e1bb2c9240e4b94d4313d9ab4bbe'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:31.191568 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-315b0fd5-2ea6-4138-af01-a243365b6fc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ad9e1bb2c9240e4b94d4313d9ab4bbe'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:31.202969 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1107/2193] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:18:30 2022] DELETE /identity/v3/users/5ad9e1bb2c9240e4b94d4313d9ab4bbe => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:18:31.207838 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb9d06-e151-4326-a2dc-f80f44d9c6ba None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:31.208156 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb9d06-e151-4326-a2dc-f80f44d9c6ba None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:31.208751 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb9d06-e151-4326-a2dc-f80f44d9c6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:31.231694 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0dcb9d06-e151-4326-a2dc-f80f44d9c6ba None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:18:31.233034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0dcb9d06-e151-4326-a2dc-f80f44d9c6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:31.293437 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:18:31.305810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1087/2194] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:18:31.318708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b0fd5-2ea6-4138-af01-a243365b6fc6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:31.320437 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-315b0fd5-2ea6-4138-af01-a243365b6fc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:31.419646 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b0fd5-2ea6-4138-af01-a243365b6fc6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:31.424146 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5e5a5-d243-4631-859f-0d8ce46803c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:31.424768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5e5a5-d243-4631-859f-0d8ce46803c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:31.425604 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5e5a5-d243-4631-859f-0d8ce46803c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:31.425886 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5e5a5-d243-4631-859f-0d8ce46803c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:31.448377 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5e5a5-d243-4631-859f-0d8ce46803c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:31.449879 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5e5a5-d243-4631-859f-0d8ce46803c4 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:31.461285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1108/2195] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:18:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:18:31.587890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb9d06-e151-4326-a2dc-f80f44d9c6ba None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:31.589517 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0dcb9d06-e151-4326-a2dc-f80f44d9c6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:31.617762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb9d06-e151-4326-a2dc-f80f44d9c6ba None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:31.621753 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f1e23-63da-4d0a-90c6-01d25207e31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:31.622579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f1e23-63da-4d0a-90c6-01d25207e31c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:31.622956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f1e23-63da-4d0a-90c6-01d25207e31c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:31.623338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f1e23-63da-4d0a-90c6-01d25207e31c admin admin] PATH_INFO: `/v3/projects/e4c58771ede14b0fae72c65ade80094d` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:31.624608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f1e23-63da-4d0a-90c6-01d25207e31c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4c58771ede14b0fae72c65ade80094d)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:31.626582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f1e23-63da-4d0a-90c6-01d25207e31c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:31.777604 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-673f1e23-63da-4d0a-90c6-01d25207e31c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4c58771ede14b0fae72c65ade80094d', 'request_id': 'req-673f1e23-63da-4d0a-90c6-01d25207e31c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:31.899971 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-673f1e23-63da-4d0a-90c6-01d25207e31c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4c58771ede14b0fae72c65ade80094d', 'request_id': 'req-673f1e23-63da-4d0a-90c6-01d25207e31c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:31.909897 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-673f1e23-63da-4d0a-90c6-01d25207e31c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4c58771ede14b0fae72c65ade80094d', 'request_id': 'req-673f1e23-63da-4d0a-90c6-01d25207e31c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:31.921982 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-673f1e23-63da-4d0a-90c6-01d25207e31c admin admin] The token cache is being invalidate because project e4c58771ede14b0fae72c65ade80094d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:18:31.922308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-673f1e23-63da-4d0a-90c6-01d25207e31c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:31.923018 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-673f1e23-63da-4d0a-90c6-01d25207e31c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:31.924549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1088/2196] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:18:31 2022] DELETE /identity/v3/projects/e4c58771ede14b0fae72c65ade80094d => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:18:31.930133 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967875d1-dd40-40e6-a858-837191888946 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:31.930452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967875d1-dd40-40e6-a858-837191888946 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:31.930731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967875d1-dd40-40e6-a858-837191888946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:31.951972 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-967875d1-dd40-40e6-a858-837191888946 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:18:31.953203 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-967875d1-dd40-40e6-a858-837191888946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:31.989352 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:18:32.032089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1109/2197] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:18:32.040861 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98c641-642c-4d38-a8fb-7ea3db5d508b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:32.041162 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98c641-642c-4d38-a8fb-7ea3db5d508b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:32.041401 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98c641-642c-4d38-a8fb-7ea3db5d508b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:32.064040 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-fb98c641-642c-4d38-a8fb-7ea3db5d508b None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:18:32.065962 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fb98c641-642c-4d38-a8fb-7ea3db5d508b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:32.094327 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1089/2198] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:18:32.100367 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967875d1-dd40-40e6-a858-837191888946 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:32.110598 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967875d1-dd40-40e6-a858-837191888946 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:32.121175 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3339b26-a8d1-40e0-a2db-9a4934bb7a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:32.121725 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3339b26-a8d1-40e0-a2db-9a4934bb7a43 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:32.121999 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3339b26-a8d1-40e0-a2db-9a4934bb7a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:32.122433 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3339b26-a8d1-40e0-a2db-9a4934bb7a43 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:32.126438 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3339b26-a8d1-40e0-a2db-9a4934bb7a43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:18:32.126438 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3339b26-a8d1-40e0-a2db-9a4934bb7a43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:32.126438 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3339b26-a8d1-40e0-a2db-9a4934bb7a43 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:32.138928 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1110/2199] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:18:32.149763 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98c641-642c-4d38-a8fb-7ea3db5d508b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:32.159747 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98c641-642c-4d38-a8fb-7ea3db5d508b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:32.166033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d2c2a-43d4-4ae5-8745-956106e5bbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:32.166716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d2c2a-43d4-4ae5-8745-956106e5bbc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:32.167137 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d2c2a-43d4-4ae5-8745-956106e5bbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:32.167411 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d2c2a-43d4-4ae5-8745-956106e5bbc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:32.168063 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d2c2a-43d4-4ae5-8745-956106e5bbc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:32.170148 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d2c2a-43d4-4ae5-8745-956106e5bbc6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:32.215714 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3339b26-a8d1-40e0-a2db-9a4934bb7a43 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:32.217577 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e3339b26-a8d1-40e0-a2db-9a4934bb7a43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:32.232076 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3339b26-a8d1-40e0-a2db-9a4934bb7a43 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:32.238806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d881be-b01e-4a90-afa0-d5e97740f7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:32.239502 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d881be-b01e-4a90-afa0-d5e97740f7ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:32.239932 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d881be-b01e-4a90-afa0-d5e97740f7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:32.240256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d881be-b01e-4a90-afa0-d5e97740f7ac admin admin] PATH_INFO: `/v3/projects/4a0e1b0bf3334765bec6efaa5b2ba49a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:32.241411 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d881be-b01e-4a90-afa0-d5e97740f7ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a0e1b0bf3334765bec6efaa5b2ba49a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:32.243683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d881be-b01e-4a90-afa0-d5e97740f7ac admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:32.279979 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1090/2200] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:18:32 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:18:32.284213 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d2c2a-43d4-4ae5-8745-956106e5bbc6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:32.350762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d2c2a-43d4-4ae5-8745-956106e5bbc6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:32.355669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45625e10-2d28-4d8b-834e-a74ae0ae6b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:32.356280 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45625e10-2d28-4d8b-834e-a74ae0ae6b8a admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:32.356551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45625e10-2d28-4d8b-834e-a74ae0ae6b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:32.356821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45625e10-2d28-4d8b-834e-a74ae0ae6b8a admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:32.357525 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45625e10-2d28-4d8b-834e-a74ae0ae6b8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:32.359389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45625e10-2d28-4d8b-834e-a74ae0ae6b8a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:32.380174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c1d881be-b01e-4a90-afa0-d5e97740f7ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a0e1b0bf3334765bec6efaa5b2ba49a', 'request_id': 'req-c1d881be-b01e-4a90-afa0-d5e97740f7ac'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:32.387450 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c1d881be-b01e-4a90-afa0-d5e97740f7ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a0e1b0bf3334765bec6efaa5b2ba49a', 'request_id': 'req-c1d881be-b01e-4a90-afa0-d5e97740f7ac'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:32.449149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c1d881be-b01e-4a90-afa0-d5e97740f7ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a0e1b0bf3334765bec6efaa5b2ba49a', 'request_id': 'req-c1d881be-b01e-4a90-afa0-d5e97740f7ac'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:32.462220 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1091/2201] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:18:32 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:18:32.464130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45625e10-2d28-4d8b-834e-a74ae0ae6b8a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:32.467219 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c1d881be-b01e-4a90-afa0-d5e97740f7ac admin admin] The token cache is being invalidate because project 4a0e1b0bf3334765bec6efaa5b2ba49a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:18:32.467702 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c1d881be-b01e-4a90-afa0-d5e97740f7ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:32.468810 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c1d881be-b01e-4a90-afa0-d5e97740f7ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:32.471733 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1111/2202] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:18:32 2022] DELETE /identity/v3/projects/4a0e1b0bf3334765bec6efaa5b2ba49a => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:18:32.479593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfb30d-ff55-40da-a32c-3853d90bdf88 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:32.486102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfb30d-ff55-40da-a32c-3853d90bdf88 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:32.486102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfb30d-ff55-40da-a32c-3853d90bdf88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:32.486310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45625e10-2d28-4d8b-834e-a74ae0ae6b8a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:32.508459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0abfb30d-ff55-40da-a32c-3853d90bdf88 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:18:32.509696 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0abfb30d-ff55-40da-a32c-3853d90bdf88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:32.531702 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:18:32.550569 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666107f5-634c-40f0-a06d-b14475f24b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:32.551178 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666107f5-634c-40f0-a06d-b14475f24b46 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:32.551458 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666107f5-634c-40f0-a06d-b14475f24b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:32.551728 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666107f5-634c-40f0-a06d-b14475f24b46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:32.552425 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666107f5-634c-40f0-a06d-b14475f24b46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:32.554249 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666107f5-634c-40f0-a06d-b14475f24b46 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:32.564173 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1092/2203] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:18:32.569017 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666107f5-634c-40f0-a06d-b14475f24b46 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:32.570242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1112/2204] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:18:32.575070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-666107f5-634c-40f0-a06d-b14475f24b46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:32.579028 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e438d-3da2-4759-95cd-076cc77c6ae7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:32.579285 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e438d-3da2-4759-95cd-076cc77c6ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:32.579582 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e438d-3da2-4759-95cd-076cc77c6ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:32.588587 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666107f5-634c-40f0-a06d-b14475f24b46 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:32.593561 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba3740-b170-4c8f-9252-d3731885aadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:32.594212 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba3740-b170-4c8f-9252-d3731885aadd admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:32.594521 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba3740-b170-4c8f-9252-d3731885aadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:32.594794 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba3740-b170-4c8f-9252-d3731885aadd admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:32.595487 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ba3740-b170-4c8f-9252-d3731885aadd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:32.597120 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ba3740-b170-4c8f-9252-d3731885aadd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:32.601841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a11e438d-3da2-4759-95cd-076cc77c6ae7 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:18:32.603021 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a11e438d-3da2-4759-95cd-076cc77c6ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:32.613995 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1093/2205] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:18:32.620616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba3740-b170-4c8f-9252-d3731885aadd admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:32.620730 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1113/2206] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:18:32.625441 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e438d-3da2-4759-95cd-076cc77c6ae7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:32.627783 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba3740-b170-4c8f-9252-d3731885aadd admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:32.633910 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e3e0aa-9a2d-4c08-b940-10b4f8aa6d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:32.634650 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3e0aa-9a2d-4c08-b940-10b4f8aa6d55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:32.635199 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3e0aa-9a2d-4c08-b940-10b4f8aa6d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:32.635538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3e0aa-9a2d-4c08-b940-10b4f8aa6d55 admin admin] PATH_INFO: `/v3/projects/0dbaa0dec7094abfaa8124b7d7b9d626/users/4c6a30ec41a3455b9ee19cb4f4dfd1f2/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:32.637483 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e438d-3da2-4759-95cd-076cc77c6ae7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:32.642996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83c28c-5860-4375-b8d7-2071d7d13dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:32.648495 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83c28c-5860-4375-b8d7-2071d7d13dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:32.648495 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83c28c-5860-4375-b8d7-2071d7d13dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:32.648495 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83c28c-5860-4375-b8d7-2071d7d13dd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:32.648495 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f83c28c-5860-4375-b8d7-2071d7d13dd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:18:32.648495 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f83c28c-5860-4375-b8d7-2071d7d13dd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:32.648495 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f83c28c-5860-4375-b8d7-2071d7d13dd7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:32.655857 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e3e0aa-9a2d-4c08-b940-10b4f8aa6d55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dbaa0dec7094abfaa8124b7d7b9d626, user_id=4c6a30ec41a3455b9ee19cb4f4dfd1f2, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:32.658879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e3e0aa-9a2d-4c08-b940-10b4f8aa6d55 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:32.664416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1114/2207] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:18:32.670641 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83c28c-5860-4375-b8d7-2071d7d13dd7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:32.681581 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83c28c-5860-4375-b8d7-2071d7d13dd7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:32.686577 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d2b39-7bc2-4a56-b7cc-cc2a628e462d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:32.687213 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d2b39-7bc2-4a56-b7cc-cc2a628e462d admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:32.687491 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d2b39-7bc2-4a56-b7cc-cc2a628e462d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:32.687760 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d2b39-7bc2-4a56-b7cc-cc2a628e462d admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:32.688408 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d2b39-7bc2-4a56-b7cc-cc2a628e462d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:32.690378 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d2b39-7bc2-4a56-b7cc-cc2a628e462d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:32.819074 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1094/2208] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:18:32 2022] PUT /identity/v3/projects/0dbaa0dec7094abfaa8124b7d7b9d626/users/4c6a30ec41a3455b9ee19cb4f4dfd1f2/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:18:32.829648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567a4b7-2e5c-44f9-a0cc-68dc42e760c3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:32.829945 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567a4b7-2e5c-44f9-a0cc-68dc42e760c3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:32.830236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567a4b7-2e5c-44f9-a0cc-68dc42e760c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:32.850129 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1115/2209] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:18:32 2022] POST /identity/v3/projects => generated 373 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:18:32.854784 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d2b39-7bc2-4a56-b7cc-cc2a628e462d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:32.863713 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:18:32.870260 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e567a4b7-2e5c-44f9-a0cc-68dc42e760c3 None None] MFA Rules not processed for user `4c6a30ec41a3455b9ee19cb4f4dfd1f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:18:32.871591 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e567a4b7-2e5c-44f9-a0cc-68dc42e760c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:32.944258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d2b39-7bc2-4a56-b7cc-cc2a628e462d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:32.949429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2fe30-5fcf-4d11-90e3-4439c1d61277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:32.953413 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2fe30-5fcf-4d11-90e3-4439c1d61277 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:32.953413 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2fe30-5fcf-4d11-90e3-4439c1d61277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:32.953413 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2fe30-5fcf-4d11-90e3-4439c1d61277 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:32.953413 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc2fe30-5fcf-4d11-90e3-4439c1d61277 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:32.953413 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc2fe30-5fcf-4d11-90e3-4439c1d61277 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:32.966735 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1095/2210] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:18:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:18:32.980428 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567a4b7-2e5c-44f9-a0cc-68dc42e760c3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:32.981992 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e567a4b7-2e5c-44f9-a0cc-68dc42e760c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:33.041674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567a4b7-2e5c-44f9-a0cc-68dc42e760c3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:33.045531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505636b-d796-456c-b4b4-12b961207e0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:33.046132 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505636b-d796-456c-b4b4-12b961207e0f service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:33.046434 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505636b-d796-456c-b4b4-12b961207e0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:33.046700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505636b-d796-456c-b4b4-12b961207e0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:33.055189 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505636b-d796-456c-b4b4-12b961207e0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:33.056599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505636b-d796-456c-b4b4-12b961207e0f service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:33.065727 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1096/2211] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:18:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:18:33.191664 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1116/2212] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:18:32 2022] POST /identity/v3/users => generated 287 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:18:33.195863 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505636b-d796-456c-b4b4-12b961207e0f service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:33.206432 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505636b-d796-456c-b4b4-12b961207e0f service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:33.211461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e279ac-9379-4579-8a43-9b78974169f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:33.212044 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e279ac-9379-4579-8a43-9b78974169f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:33.212436 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e279ac-9379-4579-8a43-9b78974169f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:33.212685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e279ac-9379-4579-8a43-9b78974169f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:33.213320 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e279ac-9379-4579-8a43-9b78974169f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:33.214957 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e279ac-9379-4579-8a43-9b78974169f6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:33.230965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1097/2213] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:18:33.236461 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2fe30-5fcf-4d11-90e3-4439c1d61277 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:33.247019 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2fe30-5fcf-4d11-90e3-4439c1d61277 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:33.251701 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52338eba-d2ea-453d-a65f-0c10bdfd1228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:33.252324 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52338eba-d2ea-453d-a65f-0c10bdfd1228 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:33.252648 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52338eba-d2ea-453d-a65f-0c10bdfd1228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:33.252920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52338eba-d2ea-453d-a65f-0c10bdfd1228 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:33.253559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52338eba-d2ea-453d-a65f-0c10bdfd1228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:33.255806 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52338eba-d2ea-453d-a65f-0c10bdfd1228 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:33.266170 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1117/2214] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:18:33.271889 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e279ac-9379-4579-8a43-9b78974169f6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:33.281972 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e279ac-9379-4579-8a43-9b78974169f6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:33.287218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a29fd7-c267-4201-8c99-07551b745a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:33.287963 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a29fd7-c267-4201-8c99-07551b745a51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:33.288334 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a29fd7-c267-4201-8c99-07551b745a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:33.288633 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a29fd7-c267-4201-8c99-07551b745a51 admin admin] PATH_INFO: `/v3/projects/c8063f17d8304cbdb382e786c0d41f36/users/3862ff77dbea4d7bb682e25c135fdd90/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:33.303712 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a29fd7-c267-4201-8c99-07551b745a51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8063f17d8304cbdb382e786c0d41f36, user_id=3862ff77dbea4d7bb682e25c135fdd90, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:33.306480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a29fd7-c267-4201-8c99-07551b745a51 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:33.472080 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1098/2215] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:18:33 2022] PUT /identity/v3/projects/c8063f17d8304cbdb382e786c0d41f36/users/3862ff77dbea4d7bb682e25c135fdd90/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:18:33.483647 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907d442-0af5-4dab-af4c-6432d86d63a2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:33.484370 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907d442-0af5-4dab-af4c-6432d86d63a2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:33.484650 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907d442-0af5-4dab-af4c-6432d86d63a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:33.523443 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4907d442-0af5-4dab-af4c-6432d86d63a2 None None] MFA Rules not processed for user `3862ff77dbea4d7bb682e25c135fdd90`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:18:33.524695 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4907d442-0af5-4dab-af4c-6432d86d63a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:33.591126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1118/2216] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:18:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:18:33.604316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a29fd7-c267-4201-8c99-07551b745a51 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:33.606973 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-02a29fd7-c267-4201-8c99-07551b745a51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:33.684855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a29fd7-c267-4201-8c99-07551b745a51 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:33.693261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60332ef0-49e4-4026-be78-4abb63fb0423 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:33.694131 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60332ef0-49e4-4026-be78-4abb63fb0423 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:33.694516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60332ef0-49e4-4026-be78-4abb63fb0423 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:33.694847 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60332ef0-49e4-4026-be78-4abb63fb0423 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:33.705241 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60332ef0-49e4-4026-be78-4abb63fb0423 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:33.706710 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60332ef0-49e4-4026-be78-4abb63fb0423 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:33.716684 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1099/2217] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:18:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:18:34.202907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:18:36.816511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907d442-0af5-4dab-af4c-6432d86d63a2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:36.890914 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907d442-0af5-4dab-af4c-6432d86d63a2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:36.900208 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516e33b-2d14-43e5-bce8-7b988b5b785d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:36.900208 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516e33b-2d14-43e5-bce8-7b988b5b785d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:36.900208 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516e33b-2d14-43e5-bce8-7b988b5b785d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:36.900208 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516e33b-2d14-43e5-bce8-7b988b5b785d admin admin] PATH_INFO: `/v3/users/4c6a30ec41a3455b9ee19cb4f4dfd1f2` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:36.900599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c516e33b-2d14-43e5-bce8-7b988b5b785d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c6a30ec41a3455b9ee19cb4f4dfd1f2)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:36.900599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c516e33b-2d14-43e5-bce8-7b988b5b785d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:36.928703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:18:37.132173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c516e33b-2d14-43e5-bce8-7b988b5b785d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c6a30ec41a3455b9ee19cb4f4dfd1f2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:37.248493 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c516e33b-2d14-43e5-bce8-7b988b5b785d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c6a30ec41a3455b9ee19cb4f4dfd1f2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:37.256927 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c516e33b-2d14-43e5-bce8-7b988b5b785d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c6a30ec41a3455b9ee19cb4f4dfd1f2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:37.270138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c516e33b-2d14-43e5-bce8-7b988b5b785d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c6a30ec41a3455b9ee19cb4f4dfd1f2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:37.271522 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c516e33b-2d14-43e5-bce8-7b988b5b785d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c6a30ec41a3455b9ee19cb4f4dfd1f2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:37.277518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1119/2218] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:18:36 2022] DELETE /identity/v3/users/4c6a30ec41a3455b9ee19cb4f4dfd1f2 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:18:37.282186 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0210ce-38a5-48ba-82e9-aa1f00f7dee6 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:37.282699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0210ce-38a5-48ba-82e9-aa1f00f7dee6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:37.282877 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0210ce-38a5-48ba-82e9-aa1f00f7dee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:37.302655 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ab0210ce-38a5-48ba-82e9-aa1f00f7dee6 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:18:37.303913 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ab0210ce-38a5-48ba-82e9-aa1f00f7dee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:37.365346 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1100/2219] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:18:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:18:37.375903 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516e33b-2d14-43e5-bce8-7b988b5b785d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:37.377342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c516e33b-2d14-43e5-bce8-7b988b5b785d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:37.434463 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516e33b-2d14-43e5-bce8-7b988b5b785d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:37.438295 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfae377-0954-4095-bbdd-beba60625f89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:37.438959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfae377-0954-4095-bbdd-beba60625f89 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:37.439242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfae377-0954-4095-bbdd-beba60625f89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:37.439511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfae377-0954-4095-bbdd-beba60625f89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:37.449425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfae377-0954-4095-bbdd-beba60625f89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:37.450896 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfae377-0954-4095-bbdd-beba60625f89 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:37.460597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1120/2220] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:18:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:18:37.889844 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0210ce-38a5-48ba-82e9-aa1f00f7dee6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:37.891285 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ab0210ce-38a5-48ba-82e9-aa1f00f7dee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:37.911392 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0210ce-38a5-48ba-82e9-aa1f00f7dee6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:37.920601 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16a437-4c4a-49ed-98da-571f5b8e837c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:37.921399 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16a437-4c4a-49ed-98da-571f5b8e837c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:37.921793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16a437-4c4a-49ed-98da-571f5b8e837c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:37.922188 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16a437-4c4a-49ed-98da-571f5b8e837c admin admin] PATH_INFO: `/v3/projects/0dbaa0dec7094abfaa8124b7d7b9d626` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:37.924215 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16a437-4c4a-49ed-98da-571f5b8e837c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dbaa0dec7094abfaa8124b7d7b9d626)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:37.926888 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16a437-4c4a-49ed-98da-571f5b8e837c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:38.094750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8f16a437-4c4a-49ed-98da-571f5b8e837c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dbaa0dec7094abfaa8124b7d7b9d626', 'request_id': 'req-8f16a437-4c4a-49ed-98da-571f5b8e837c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:38.192689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8f16a437-4c4a-49ed-98da-571f5b8e837c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dbaa0dec7094abfaa8124b7d7b9d626', 'request_id': 'req-8f16a437-4c4a-49ed-98da-571f5b8e837c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:38.201275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8f16a437-4c4a-49ed-98da-571f5b8e837c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dbaa0dec7094abfaa8124b7d7b9d626', 'request_id': 'req-8f16a437-4c4a-49ed-98da-571f5b8e837c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:38.213406 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8f16a437-4c4a-49ed-98da-571f5b8e837c admin admin] The token cache is being invalidate because project 0dbaa0dec7094abfaa8124b7d7b9d626 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:18:38.213806 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8f16a437-4c4a-49ed-98da-571f5b8e837c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:38.214605 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8f16a437-4c4a-49ed-98da-571f5b8e837c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:38.216560 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1101/2221] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:18:37 2022] DELETE /identity/v3/projects/0dbaa0dec7094abfaa8124b7d7b9d626 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:18:40.181607 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7e5cd-0c0f-4efd-84b9-8611e01b8162 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:40.182262 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7e5cd-0c0f-4efd-84b9-8611e01b8162 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:40.182669 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7e5cd-0c0f-4efd-84b9-8611e01b8162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:40.222323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-50e7e5cd-0c0f-4efd-84b9-8611e01b8162 None None] MFA Rules not processed for user `4f9e1767ec90435f80248a2930081265`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:18:40.224601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-50e7e5cd-0c0f-4efd-84b9-8611e01b8162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:40.317234 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1121/2222] 10.222.0.45 () {58 vars in 967 bytes} [Wed Aug 10 17:18:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:18:40.357566 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16a437-4c4a-49ed-98da-571f5b8e837c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:40.359721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8f16a437-4c4a-49ed-98da-571f5b8e837c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:40.406715 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:18:40.468102 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16a437-4c4a-49ed-98da-571f5b8e837c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:40.470299 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd0c89-b591-4711-9b04-175cd369dffa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4986821ef9f84b99a835e387635aa9a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:40.471012 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd0c89-b591-4711-9b04-175cd369dffa service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:40.471360 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd0c89-b591-4711-9b04-175cd369dffa service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:40.471704 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd0c89-b591-4711-9b04-175cd369dffa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:40.482771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd0c89-b591-4711-9b04-175cd369dffa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:40.484674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd0c89-b591-4711-9b04-175cd369dffa service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:40.494752 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1102/2223] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 10 17:18:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 17:18:46.438992 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7e5cd-0c0f-4efd-84b9-8611e01b8162 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:46.440499 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-50e7e5cd-0c0f-4efd-84b9-8611e01b8162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:46.495494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:18:46.526434 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7e5cd-0c0f-4efd-84b9-8611e01b8162 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:46.531191 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908f4e1-f9b2-4254-ba58-dcabef354388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:46.533476 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908f4e1-f9b2-4254-ba58-dcabef354388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:46.533476 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908f4e1-f9b2-4254-ba58-dcabef354388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:46.533476 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908f4e1-f9b2-4254-ba58-dcabef354388 admin admin] PATH_INFO: `/v3/users/3862ff77dbea4d7bb682e25c135fdd90` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:46.533476 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a908f4e1-f9b2-4254-ba58-dcabef354388 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3862ff77dbea4d7bb682e25c135fdd90)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:46.535558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a908f4e1-f9b2-4254-ba58-dcabef354388 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:46.623074 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8ed54-3095-40c6-ad34-06ac7558ffd2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:46.623605 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8ed54-3095-40c6-ad34-06ac7558ffd2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:46.623605 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8ed54-3095-40c6-ad34-06ac7558ffd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:46.643302 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b2e8ed54-3095-40c6-ad34-06ac7558ffd2 None None] MFA Rules not processed for user `48feb2fc02f04e739c5e1bf7d68baa16`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:18:46.644520 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b2e8ed54-3095-40c6-ad34-06ac7558ffd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:46.701631 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1103/2224] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:18:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:18:46.796893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a908f4e1-f9b2-4254-ba58-dcabef354388 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3862ff77dbea4d7bb682e25c135fdd90'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:46.968433 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a908f4e1-f9b2-4254-ba58-dcabef354388 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3862ff77dbea4d7bb682e25c135fdd90'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:46.977989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a908f4e1-f9b2-4254-ba58-dcabef354388 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3862ff77dbea4d7bb682e25c135fdd90'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:46.991373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a908f4e1-f9b2-4254-ba58-dcabef354388 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3862ff77dbea4d7bb682e25c135fdd90'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:46.992117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a908f4e1-f9b2-4254-ba58-dcabef354388 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3862ff77dbea4d7bb682e25c135fdd90'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:46.998271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1122/2225] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:18:46 2022] DELETE /identity/v3/users/3862ff77dbea4d7bb682e25c135fdd90 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:18:47.002917 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77afee6-9649-4f41-a085-bdb9b9ae176d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:47.003439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77afee6-9649-4f41-a085-bdb9b9ae176d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:47.003717 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77afee6-9649-4f41-a085-bdb9b9ae176d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:47.023269 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b77afee6-9649-4f41-a085-bdb9b9ae176d None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:18:47.024486 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b77afee6-9649-4f41-a085-bdb9b9ae176d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:47.087369 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1104/2226] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:18:47.097940 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908f4e1-f9b2-4254-ba58-dcabef354388 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:47.099925 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a908f4e1-f9b2-4254-ba58-dcabef354388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:47.166057 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908f4e1-f9b2-4254-ba58-dcabef354388 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:47.169991 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f9910-70af-40e4-b28b-ac85d71f0381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:47.170646 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f9910-70af-40e4-b28b-ac85d71f0381 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:47.170922 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f9910-70af-40e4-b28b-ac85d71f0381 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:47.171189 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f9910-70af-40e4-b28b-ac85d71f0381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:47.181617 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f9910-70af-40e4-b28b-ac85d71f0381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:47.183184 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f9910-70af-40e4-b28b-ac85d71f0381 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:47.193454 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1123/2227] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:18:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:18:47.748165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77afee6-9649-4f41-a085-bdb9b9ae176d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:47.750641 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b77afee6-9649-4f41-a085-bdb9b9ae176d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:47.773995 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77afee6-9649-4f41-a085-bdb9b9ae176d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:47.780515 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28411b60-1ed2-42e4-ba9e-517c39488539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:47.782090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28411b60-1ed2-42e4-ba9e-517c39488539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:47.782539 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28411b60-1ed2-42e4-ba9e-517c39488539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:47.782685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28411b60-1ed2-42e4-ba9e-517c39488539 admin admin] PATH_INFO: `/v3/projects/c8063f17d8304cbdb382e786c0d41f36` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:47.784384 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28411b60-1ed2-42e4-ba9e-517c39488539 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8063f17d8304cbdb382e786c0d41f36)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:47.786781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28411b60-1ed2-42e4-ba9e-517c39488539 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:47.946856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-28411b60-1ed2-42e4-ba9e-517c39488539 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8063f17d8304cbdb382e786c0d41f36', 'request_id': 'req-28411b60-1ed2-42e4-ba9e-517c39488539'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:47.950687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:18:48.060170 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-28411b60-1ed2-42e4-ba9e-517c39488539 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8063f17d8304cbdb382e786c0d41f36', 'request_id': 'req-28411b60-1ed2-42e4-ba9e-517c39488539'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:48.067086 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-28411b60-1ed2-42e4-ba9e-517c39488539 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8063f17d8304cbdb382e786c0d41f36', 'request_id': 'req-28411b60-1ed2-42e4-ba9e-517c39488539'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:48.078339 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-28411b60-1ed2-42e4-ba9e-517c39488539 admin admin] The token cache is being invalidate because project c8063f17d8304cbdb382e786c0d41f36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:18:48.080606 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-28411b60-1ed2-42e4-ba9e-517c39488539 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:48.080606 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-28411b60-1ed2-42e4-ba9e-517c39488539 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:18:48.081358 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1105/2228] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:18:47 2022] DELETE /identity/v3/projects/c8063f17d8304cbdb382e786c0d41f36 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:18:48.087043 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1f79a-62c2-4c3d-b09c-aad02c8b47f2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:48.087551 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1f79a-62c2-4c3d-b09c-aad02c8b47f2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:48.088029 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1f79a-62c2-4c3d-b09c-aad02c8b47f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:48.109561 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-40b1f79a-62c2-4c3d-b09c-aad02c8b47f2 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:18:48.110894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-40b1f79a-62c2-4c3d-b09c-aad02c8b47f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:48.164602 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:18:48.173257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1124/2229] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:18:48.182625 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ba07e-a2ff-49c8-9610-8fb7976f428f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:48.182900 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ba07e-a2ff-49c8-9610-8fb7976f428f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:48.183166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ba07e-a2ff-49c8-9610-8fb7976f428f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:48.206298 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3d2ba07e-a2ff-49c8-9610-8fb7976f428f None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:18:48.207567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3d2ba07e-a2ff-49c8-9610-8fb7976f428f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:48.217041 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1106/2230] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:18:48.221509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1f79a-62c2-4c3d-b09c-aad02c8b47f2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:48.231913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1f79a-62c2-4c3d-b09c-aad02c8b47f2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:48.237325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06318087-3e0f-4d83-8445-2b2c246359a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:48.237969 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06318087-3e0f-4d83-8445-2b2c246359a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:48.238244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06318087-3e0f-4d83-8445-2b2c246359a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:48.238551 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06318087-3e0f-4d83-8445-2b2c246359a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:48.239270 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06318087-3e0f-4d83-8445-2b2c246359a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:18:48.239580 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06318087-3e0f-4d83-8445-2b2c246359a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:48.241199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06318087-3e0f-4d83-8445-2b2c246359a7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:48.255384 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1125/2231] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:18:48.260700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ba07e-a2ff-49c8-9610-8fb7976f428f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:48.280997 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ba07e-a2ff-49c8-9610-8fb7976f428f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:48.280997 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f713bdfc-a401-498a-b3a3-498096b57eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:48.280997 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713bdfc-a401-498a-b3a3-498096b57eaf admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:48.280997 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713bdfc-a401-498a-b3a3-498096b57eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:48.281487 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713bdfc-a401-498a-b3a3-498096b57eaf admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:48.281487 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f713bdfc-a401-498a-b3a3-498096b57eaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:48.281487 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f713bdfc-a401-498a-b3a3-498096b57eaf admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:48.455497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1107/2232] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:18:48 2022] POST /identity/v3/projects => generated 373 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:18:48.462529 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06318087-3e0f-4d83-8445-2b2c246359a7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:48.479766 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06318087-3e0f-4d83-8445-2b2c246359a7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:48.539330 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52899a-7675-4c1e-8599-af694f4c0aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:48.539882 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52899a-7675-4c1e-8599-af694f4c0aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:48.540161 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52899a-7675-4c1e-8599-af694f4c0aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:48.540518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52899a-7675-4c1e-8599-af694f4c0aa0 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:48.541054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf52899a-7675-4c1e-8599-af694f4c0aa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:48.543043 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf52899a-7675-4c1e-8599-af694f4c0aa0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:48.775314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1126/2233] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:18:48 2022] POST /identity/v3/users => generated 287 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:18:48.779747 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f713bdfc-a401-498a-b3a3-498096b57eaf admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:48.791409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f713bdfc-a401-498a-b3a3-498096b57eaf admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:48.796326 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d31ef-f735-4168-a410-140a0d84c21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:48.799782 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d31ef-f735-4168-a410-140a0d84c21f admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:48.799782 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d31ef-f735-4168-a410-140a0d84c21f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:48.799782 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d31ef-f735-4168-a410-140a0d84c21f admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:48.799782 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d31ef-f735-4168-a410-140a0d84c21f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:48.800196 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d31ef-f735-4168-a410-140a0d84c21f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:48.813892 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1108/2234] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:18:48.819757 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52899a-7675-4c1e-8599-af694f4c0aa0 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:48.834709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52899a-7675-4c1e-8599-af694f4c0aa0 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:48.840316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727361b7-2791-4ef4-95d7-68502524be45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:48.840982 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727361b7-2791-4ef4-95d7-68502524be45 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:48.841341 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727361b7-2791-4ef4-95d7-68502524be45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:48.841849 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727361b7-2791-4ef4-95d7-68502524be45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:48.842743 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727361b7-2791-4ef4-95d7-68502524be45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:48.844576 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727361b7-2791-4ef4-95d7-68502524be45 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:48.858166 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1127/2235] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:18:48.862588 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d31ef-f735-4168-a410-140a0d84c21f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:48.874061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d31ef-f735-4168-a410-140a0d84c21f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:48.879192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a1a981-d36e-4546-9d79-aacd8702d667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:48.880452 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a1a981-d36e-4546-9d79-aacd8702d667 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:48.880794 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a1a981-d36e-4546-9d79-aacd8702d667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:48.881205 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a1a981-d36e-4546-9d79-aacd8702d667 admin admin] PATH_INFO: `/v3/projects/8dd5034c1fae4e369eb74852eb185468/users/baa1fc74f5fd48a7846d44337374d480/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:48.897872 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a1a981-d36e-4546-9d79-aacd8702d667 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dd5034c1fae4e369eb74852eb185468, user_id=baa1fc74f5fd48a7846d44337374d480, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:48.900587 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a1a981-d36e-4546-9d79-aacd8702d667 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:49.019987 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1109/2236] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:18:48 2022] PUT /identity/v3/projects/8dd5034c1fae4e369eb74852eb185468/users/baa1fc74f5fd48a7846d44337374d480/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:18:49.032514 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c16e81-3998-4ca3-8b04-9b656e7d1a42 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:49.032800 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c16e81-3998-4ca3-8b04-9b656e7d1a42 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:49.033071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c16e81-3998-4ca3-8b04-9b656e7d1a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:49.061598 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-95c16e81-3998-4ca3-8b04-9b656e7d1a42 None None] MFA Rules not processed for user `baa1fc74f5fd48a7846d44337374d480`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:18:49.062793 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-95c16e81-3998-4ca3-8b04-9b656e7d1a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:49.131900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1128/2237] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:18:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:18:49.143125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a1a981-d36e-4546-9d79-aacd8702d667 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:18:49.144869 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d7a1a981-d36e-4546-9d79-aacd8702d667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:49.214118 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a1a981-d36e-4546-9d79-aacd8702d667 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:18:49.220166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdf7b1-7920-4083-953c-f38bc873500e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:18:49.221239 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdf7b1-7920-4083-953c-f38bc873500e service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:49.221713 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdf7b1-7920-4083-953c-f38bc873500e service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:49.222193 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdf7b1-7920-4083-953c-f38bc873500e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:49.232071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbdf7b1-7920-4083-953c-f38bc873500e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:18:49.233505 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbdf7b1-7920-4083-953c-f38bc873500e service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:18:49.244347 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1110/2238] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:18:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:18:51.075582 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a6cda-9406-408f-a5d1-fbac20da2e31 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:18:51.076116 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a6cda-9406-408f-a5d1-fbac20da2e31 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:18:51.076116 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a6cda-9406-408f-a5d1-fbac20da2e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:18:51.095827 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1f7a6cda-9406-408f-a5d1-fbac20da2e31 None None] MFA Rules not processed for user `baa1fc74f5fd48a7846d44337374d480`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:18:51.098966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1f7a6cda-9406-408f-a5d1-fbac20da2e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:18:51.104577 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1129/2239] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:18:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:18:59.238633 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=59,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:19:01.090042 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=75,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:19:02.920216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdf7b1-7920-4083-953c-f38bc873500e service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:02.921803 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9bbdf7b1-7920-4083-953c-f38bc873500e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:02.982720 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdf7b1-7920-4083-953c-f38bc873500e service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:02.986377 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957aa4ad-218d-42a3-a02e-249ad7f0fa7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:02.987104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957aa4ad-218d-42a3-a02e-249ad7f0fa7b service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:02.987391 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957aa4ad-218d-42a3-a02e-249ad7f0fa7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:02.987635 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957aa4ad-218d-42a3-a02e-249ad7f0fa7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:02.989573 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-957aa4ad-218d-42a3-a02e-249ad7f0fa7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:03.065610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957aa4ad-218d-42a3-a02e-249ad7f0fa7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:03.067095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957aa4ad-218d-42a3-a02e-249ad7f0fa7b service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:03.079165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1111/2240] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:19:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:19:04.899373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a6cda-9406-408f-a5d1-fbac20da2e31 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:04.916521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a6cda-9406-408f-a5d1-fbac20da2e31 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:04.923099 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e097af1f-8399-4bd1-9ea2-980df3358864 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:04.924123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097af1f-8399-4bd1-9ea2-980df3358864 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:04.924866 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097af1f-8399-4bd1-9ea2-980df3358864 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:04.925347 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097af1f-8399-4bd1-9ea2-980df3358864 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:04.928187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e097af1f-8399-4bd1-9ea2-980df3358864 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:04.950750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e097af1f-8399-4bd1-9ea2-980df3358864 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:04.952222 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e097af1f-8399-4bd1-9ea2-980df3358864 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:04.966207 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1130/2241] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:19:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:19:06.429923 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e8b1b-1e3b-4a99-9659-e73807f755ab None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:06.430643 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e8b1b-1e3b-4a99-9659-e73807f755ab None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:06.430643 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e8b1b-1e3b-4a99-9659-e73807f755ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:06.450294 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e10e8b1b-1e3b-4a99-9659-e73807f755ab None None] MFA Rules not processed for user `7d2a65984c4f42fbb18977c7345c42ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:19:06.451613 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e10e8b1b-1e3b-4a99-9659-e73807f755ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:06.525047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1112/2242] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:19:06 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:19:14.174022 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b66383-642c-4d1c-98a7-d280a7a6b517 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:14.174566 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b66383-642c-4d1c-98a7-d280a7a6b517 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:14.174670 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b66383-642c-4d1c-98a7-d280a7a6b517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:14.194769 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d2b66383-642c-4d1c-98a7-d280a7a6b517 None None] MFA Rules not processed for user `4f9e1767ec90435f80248a2930081265`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:19:14.195999 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d2b66383-642c-4d1c-98a7-d280a7a6b517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:14.264855 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1131/2243] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 10 17:19:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:19:14.280032 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e8b1b-1e3b-4a99-9659-e73807f755ab None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:14.281773 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e10e8b1b-1e3b-4a99-9659-e73807f755ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:14.358449 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:19:14.383177 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e8b1b-1e3b-4a99-9659-e73807f755ab None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:14.388650 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd85bb-86d8-4b25-b8ca-2535f3c0b5f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4986821ef9f84b99a835e387635aa9a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:14.389263 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd85bb-86d8-4b25-b8ca-2535f3c0b5f6 service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:14.389538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd85bb-86d8-4b25-b8ca-2535f3c0b5f6 service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:14.389805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd85bb-86d8-4b25-b8ca-2535f3c0b5f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:14.399607 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cd85bb-86d8-4b25-b8ca-2535f3c0b5f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:14.401214 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cd85bb-86d8-4b25-b8ca-2535f3c0b5f6 service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:14.423860 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1113/2244] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 10 17:19:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 17:19:15.584844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4626fc4-a57b-470f-af19-1aa8187e9582 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:15.585375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4626fc4-a57b-470f-af19-1aa8187e9582 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:15.585375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4626fc4-a57b-470f-af19-1aa8187e9582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:15.605060 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a4626fc4-a57b-470f-af19-1aa8187e9582 None None] MFA Rules not processed for user `4f9e1767ec90435f80248a2930081265`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:19:15.606476 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a4626fc4-a57b-470f-af19-1aa8187e9582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:15.615716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1132/2245] 10.222.0.45 () {58 vars in 967 bytes} [Wed Aug 10 17:19:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:19:15.631577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd85bb-86d8-4b25-b8ca-2535f3c0b5f6 service swift] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:15.644758 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd85bb-86d8-4b25-b8ca-2535f3c0b5f6 service swift] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:15.650678 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbece7a-0012-4b82-a722-69ac91cf27bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4986821ef9f84b99a835e387635aa9a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:15.651320 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbece7a-0012-4b82-a722-69ac91cf27bb service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:15.651598 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbece7a-0012-4b82-a722-69ac91cf27bb service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:15.652019 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbece7a-0012-4b82-a722-69ac91cf27bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:15.660979 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbece7a-0012-4b82-a722-69ac91cf27bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:15.662633 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbece7a-0012-4b82-a722-69ac91cf27bb service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:15.672683 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1114/2246] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 10 17:19:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 17:19:16.373418 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1bd0b-9de0-4770-b8dc-3147130c9b39 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:16.373823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1bd0b-9de0-4770-b8dc-3147130c9b39 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:16.374160 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1bd0b-9de0-4770-b8dc-3147130c9b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:16.396093 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1fb1bd0b-9de0-4770-b8dc-3147130c9b39 None None] MFA Rules not processed for user `4f9e1767ec90435f80248a2930081265`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:19:16.397291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1fb1bd0b-9de0-4770-b8dc-3147130c9b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:16.404520 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1133/2247] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 10 17:19:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:19:16.417277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbece7a-0012-4b82-a722-69ac91cf27bb service swift] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:16.427914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbece7a-0012-4b82-a722-69ac91cf27bb service swift] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:16.433060 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a050a-5b2f-40a6-92fa-c08fc2f3fca5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4986821ef9f84b99a835e387635aa9a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:16.433674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a050a-5b2f-40a6-92fa-c08fc2f3fca5 service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:16.433951 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a050a-5b2f-40a6-92fa-c08fc2f3fca5 service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:16.434227 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a050a-5b2f-40a6-92fa-c08fc2f3fca5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:16.443428 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a050a-5b2f-40a6-92fa-c08fc2f3fca5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:16.444996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a050a-5b2f-40a6-92fa-c08fc2f3fca5 service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:16.453461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1115/2248] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 10 17:19:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 17:19:17.107590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2eafa3-ce7a-4747-8457-991e8c13da7e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:17.108030 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2eafa3-ce7a-4747-8457-991e8c13da7e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:17.108118 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2eafa3-ce7a-4747-8457-991e8c13da7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:17.130770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3d2eafa3-ce7a-4747-8457-991e8c13da7e None None] MFA Rules not processed for user `4f9e1767ec90435f80248a2930081265`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:19:17.131948 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3d2eafa3-ce7a-4747-8457-991e8c13da7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:17.139537 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1134/2249] 10.222.0.45 () {58 vars in 967 bytes} [Wed Aug 10 17:19:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:19:17.152145 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a050a-5b2f-40a6-92fa-c08fc2f3fca5 service swift] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:17.163658 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a050a-5b2f-40a6-92fa-c08fc2f3fca5 service swift] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:17.168929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2eb1c-5787-44cd-9411-75025dd78c25 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4986821ef9f84b99a835e387635aa9a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:17.169524 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2eb1c-5787-44cd-9411-75025dd78c25 service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:17.169795 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2eb1c-5787-44cd-9411-75025dd78c25 service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:17.170059 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2eb1c-5787-44cd-9411-75025dd78c25 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:17.179428 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2eb1c-5787-44cd-9411-75025dd78c25 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:17.180991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2eb1c-5787-44cd-9411-75025dd78c25 service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:17.189308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1116/2250] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 10 17:19:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 17:19:20.007920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2eafa3-ce7a-4747-8457-991e8c13da7e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:20.071729 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2eafa3-ce7a-4747-8457-991e8c13da7e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:20.076458 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d49e2d-5aee-42d7-8c0c-e0142b2aeb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:20.077077 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d49e2d-5aee-42d7-8c0c-e0142b2aeb2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:20.077355 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d49e2d-5aee-42d7-8c0c-e0142b2aeb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:20.077625 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d49e2d-5aee-42d7-8c0c-e0142b2aeb2a admin admin] PATH_INFO: `/v3/users/baa1fc74f5fd48a7846d44337374d480` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:20.078770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d49e2d-5aee-42d7-8c0c-e0142b2aeb2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=baa1fc74f5fd48a7846d44337374d480)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:20.080779 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d49e2d-5aee-42d7-8c0c-e0142b2aeb2a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:20.261824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-96d49e2d-5aee-42d7-8c0c-e0142b2aeb2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baa1fc74f5fd48a7846d44337374d480'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:19:20.338631 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-96d49e2d-5aee-42d7-8c0c-e0142b2aeb2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baa1fc74f5fd48a7846d44337374d480'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:19:20.346458 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-96d49e2d-5aee-42d7-8c0c-e0142b2aeb2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baa1fc74f5fd48a7846d44337374d480'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:19:20.354977 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:19:20.359568 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-96d49e2d-5aee-42d7-8c0c-e0142b2aeb2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baa1fc74f5fd48a7846d44337374d480'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:19:20.360279 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-96d49e2d-5aee-42d7-8c0c-e0142b2aeb2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baa1fc74f5fd48a7846d44337374d480'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:19:20.366993 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1135/2251] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:19:20 2022] DELETE /identity/v3/users/baa1fc74f5fd48a7846d44337374d480 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:19:20.371488 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfe379-86f7-46a8-a070-f4a6f37eebd2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:20.371969 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfe379-86f7-46a8-a070-f4a6f37eebd2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:20.380965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfe379-86f7-46a8-a070-f4a6f37eebd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:20.392673 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-10bfe379-86f7-46a8-a070-f4a6f37eebd2 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:19:20.393883 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-10bfe379-86f7-46a8-a070-f4a6f37eebd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:20.455351 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1117/2252] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:19:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:19:20.462931 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d49e2d-5aee-42d7-8c0c-e0142b2aeb2a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:20.464447 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-96d49e2d-5aee-42d7-8c0c-e0142b2aeb2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:20.523780 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d49e2d-5aee-42d7-8c0c-e0142b2aeb2a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:20.527427 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e3f05-b69a-4d99-8466-d96279ba5e79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:20.528128 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e3f05-b69a-4d99-8466-d96279ba5e79 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:20.528329 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e3f05-b69a-4d99-8466-d96279ba5e79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:20.528597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e3f05-b69a-4d99-8466-d96279ba5e79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:20.538797 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2e3f05-b69a-4d99-8466-d96279ba5e79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:20.540748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2e3f05-b69a-4d99-8466-d96279ba5e79 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:20.550479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1136/2253] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:19:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:19:20.820153 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfe379-86f7-46a8-a070-f4a6f37eebd2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:20.821862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-10bfe379-86f7-46a8-a070-f4a6f37eebd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:20.838070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfe379-86f7-46a8-a070-f4a6f37eebd2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:20.843768 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df3a4a-1f76-42e5-85cd-516803801628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:20.844550 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df3a4a-1f76-42e5-85cd-516803801628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:20.844896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df3a4a-1f76-42e5-85cd-516803801628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:20.845054 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df3a4a-1f76-42e5-85cd-516803801628 admin admin] PATH_INFO: `/v3/projects/8dd5034c1fae4e369eb74852eb185468` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:20.846481 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97df3a4a-1f76-42e5-85cd-516803801628 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dd5034c1fae4e369eb74852eb185468)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:20.849031 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97df3a4a-1f76-42e5-85cd-516803801628 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:20.974622 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-97df3a4a-1f76-42e5-85cd-516803801628 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dd5034c1fae4e369eb74852eb185468', 'request_id': 'req-97df3a4a-1f76-42e5-85cd-516803801628'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:19:21.058181 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-97df3a4a-1f76-42e5-85cd-516803801628 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dd5034c1fae4e369eb74852eb185468', 'request_id': 'req-97df3a4a-1f76-42e5-85cd-516803801628'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:19:21.064936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-97df3a4a-1f76-42e5-85cd-516803801628 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dd5034c1fae4e369eb74852eb185468', 'request_id': 'req-97df3a4a-1f76-42e5-85cd-516803801628'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:19:21.075322 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-97df3a4a-1f76-42e5-85cd-516803801628 admin admin] The token cache is being invalidate because project 8dd5034c1fae4e369eb74852eb185468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:19:21.075610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-97df3a4a-1f76-42e5-85cd-516803801628 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:19:21.076235 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-97df3a4a-1f76-42e5-85cd-516803801628 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:19:21.077738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1118/2254] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:19:20 2022] DELETE /identity/v3/projects/8dd5034c1fae4e369eb74852eb185468 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:19:21.083377 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ffef4-9ab4-4ce3-85f4-a108294a7568 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:21.083861 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ffef4-9ab4-4ce3-85f4-a108294a7568 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:21.083957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ffef4-9ab4-4ce3-85f4-a108294a7568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:21.104190 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9a4ffef4-9ab4-4ce3-85f4-a108294a7568 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:19:21.105395 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9a4ffef4-9ab4-4ce3-85f4-a108294a7568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:21.165707 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1137/2255] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:19:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:19:21.173493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be004d-23c9-4b1e-a791-f493412b1d33 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:21.173827 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be004d-23c9-4b1e-a791-f493412b1d33 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:21.174101 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be004d-23c9-4b1e-a791-f493412b1d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:21.194645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-10be004d-23c9-4b1e-a791-f493412b1d33 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:19:21.195994 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-10be004d-23c9-4b1e-a791-f493412b1d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:21.204384 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1119/2256] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:19:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:19:21.209392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ffef4-9ab4-4ce3-85f4-a108294a7568 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:21.219112 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ffef4-9ab4-4ce3-85f4-a108294a7568 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:21.224544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61a81e-5689-48f9-be9e-bd57379bd2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:21.225141 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61a81e-5689-48f9-be9e-bd57379bd2b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:21.225414 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61a81e-5689-48f9-be9e-bd57379bd2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:21.225679 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61a81e-5689-48f9-be9e-bd57379bd2b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:21.226374 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61a81e-5689-48f9-be9e-bd57379bd2b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:19:21.226668 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61a81e-5689-48f9-be9e-bd57379bd2b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:21.228271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61a81e-5689-48f9-be9e-bd57379bd2b9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:21.239985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1138/2257] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:19:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:19:21.249032 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be004d-23c9-4b1e-a791-f493412b1d33 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:21.260290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be004d-23c9-4b1e-a791-f493412b1d33 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:21.265200 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea411324-dcec-487e-a3bf-e253854ef0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:21.265797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea411324-dcec-487e-a3bf-e253854ef0a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:21.266070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea411324-dcec-487e-a3bf-e253854ef0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:21.266338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea411324-dcec-487e-a3bf-e253854ef0a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:21.267065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea411324-dcec-487e-a3bf-e253854ef0a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:21.269282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea411324-dcec-487e-a3bf-e253854ef0a0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:21.281716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:19:21.352338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1120/2258] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:19:21 2022] POST /identity/v3/projects => generated 389 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:19:21.356375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61a81e-5689-48f9-be9e-bd57379bd2b9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:21.367459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61a81e-5689-48f9-be9e-bd57379bd2b9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:21.425556 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28064cb5-ff12-4b37-a1aa-d220f942e879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:21.426444 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28064cb5-ff12-4b37-a1aa-d220f942e879 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:21.426810 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28064cb5-ff12-4b37-a1aa-d220f942e879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:21.427091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28064cb5-ff12-4b37-a1aa-d220f942e879 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:21.428315 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28064cb5-ff12-4b37-a1aa-d220f942e879 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:21.430245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28064cb5-ff12-4b37-a1aa-d220f942e879 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:21.538418 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1139/2259] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:19:21 2022] POST /identity/v3/users => generated 295 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:19:21.542581 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea411324-dcec-487e-a3bf-e253854ef0a0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:21.553259 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea411324-dcec-487e-a3bf-e253854ef0a0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:21.557922 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e8c5c-2e48-4646-8c52-d3c7c6448629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:21.558545 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e8c5c-2e48-4646-8c52-d3c7c6448629 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:21.558822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e8c5c-2e48-4646-8c52-d3c7c6448629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:21.559093 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e8c5c-2e48-4646-8c52-d3c7c6448629 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:21.559736 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e8c5c-2e48-4646-8c52-d3c7c6448629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:21.561330 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e8c5c-2e48-4646-8c52-d3c7c6448629 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:21.572089 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1121/2260] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:19:21.576085 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28064cb5-ff12-4b37-a1aa-d220f942e879 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:21.586987 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28064cb5-ff12-4b37-a1aa-d220f942e879 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:21.591673 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7b5f0-60d0-44bc-b12f-041550e18200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:21.592263 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7b5f0-60d0-44bc-b12f-041550e18200 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:21.592541 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7b5f0-60d0-44bc-b12f-041550e18200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:21.592813 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7b5f0-60d0-44bc-b12f-041550e18200 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:21.593476 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7b5f0-60d0-44bc-b12f-041550e18200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:21.595108 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7b5f0-60d0-44bc-b12f-041550e18200 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:21.600497 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:19:21.611796 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1140/2261] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:19:21.615589 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e8c5c-2e48-4646-8c52-d3c7c6448629 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:21.625764 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e8c5c-2e48-4646-8c52-d3c7c6448629 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:21.631915 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e567d-2952-4c32-9d84-70a1d14ad510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:21.632560 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e567d-2952-4c32-9d84-70a1d14ad510 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:21.632840 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e567d-2952-4c32-9d84-70a1d14ad510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:21.633111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e567d-2952-4c32-9d84-70a1d14ad510 admin admin] PATH_INFO: `/v3/projects/d5ad37fbf76a4d0da32e49247c5cf468/users/a51c40439edb4e2caf50da73865161f1/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:21.648642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2e567d-2952-4c32-9d84-70a1d14ad510 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5ad37fbf76a4d0da32e49247c5cf468, user_id=a51c40439edb4e2caf50da73865161f1, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:21.651532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2e567d-2952-4c32-9d84-70a1d14ad510 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:21.742575 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1122/2262] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:19:21 2022] PUT /identity/v3/projects/d5ad37fbf76a4d0da32e49247c5cf468/users/a51c40439edb4e2caf50da73865161f1/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:19:21.753582 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dca93-4446-4732-a744-046036cdbe94 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:21.753864 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dca93-4446-4732-a744-046036cdbe94 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:21.754868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dca93-4446-4732-a744-046036cdbe94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:21.782627 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7c2dca93-4446-4732-a744-046036cdbe94 None None] MFA Rules not processed for user `a51c40439edb4e2caf50da73865161f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:19:21.783859 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7c2dca93-4446-4732-a744-046036cdbe94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:21.848087 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1141/2263] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:19:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:19:21.859450 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e567d-2952-4c32-9d84-70a1d14ad510 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:21.861789 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ac2e567d-2952-4c32-9d84-70a1d14ad510 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:21.932368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e567d-2952-4c32-9d84-70a1d14ad510 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:21.936178 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c7995-dc7f-4a08-b7f9-741214775f5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:21.941144 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c7995-dc7f-4a08-b7f9-741214775f5c service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:21.941144 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c7995-dc7f-4a08-b7f9-741214775f5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:21.941144 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c7995-dc7f-4a08-b7f9-741214775f5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:21.946720 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c7995-dc7f-4a08-b7f9-741214775f5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:21.948164 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c7995-dc7f-4a08-b7f9-741214775f5c service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:21.958663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1123/2264] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:19:21 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:19:31.844964 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=32 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:19:31.953806 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=44,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:19:33.768828 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2dca93-4446-4732-a744-046036cdbe94 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:33.770383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7c2dca93-4446-4732-a744-046036cdbe94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:33.844168 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2dca93-4446-4732-a744-046036cdbe94 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:33.849674 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85407d8-b9ec-429e-8d07-0843a862ed8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:33.850599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85407d8-b9ec-429e-8d07-0843a862ed8a service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:33.850866 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85407d8-b9ec-429e-8d07-0843a862ed8a service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:33.851139 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85407d8-b9ec-429e-8d07-0843a862ed8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:33.852867 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e85407d8-b9ec-429e-8d07-0843a862ed8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:33.910960 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85407d8-b9ec-429e-8d07-0843a862ed8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:33.913091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85407d8-b9ec-429e-8d07-0843a862ed8a service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:33.922060 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1142/2265] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:19:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:19:34.385612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c7995-dc7f-4a08-b7f9-741214775f5c service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:34.387089 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-157c7995-dc7f-4a08-b7f9-741214775f5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:34.403326 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c7995-dc7f-4a08-b7f9-741214775f5c service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:34.408777 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69654219-d630-4b91-86e4-c8bba78fb987 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:34.409365 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69654219-d630-4b91-86e4-c8bba78fb987 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:34.409645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69654219-d630-4b91-86e4-c8bba78fb987 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:34.409917 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69654219-d630-4b91-86e4-c8bba78fb987 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:34.411652 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-69654219-d630-4b91-86e4-c8bba78fb987 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:34.471552 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69654219-d630-4b91-86e4-c8bba78fb987 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:34.473171 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69654219-d630-4b91-86e4-c8bba78fb987 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:34.483692 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1124/2266] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:19:34 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:19:43.277649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85407d8-b9ec-429e-8d07-0843a862ed8a service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:43.342404 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85407d8-b9ec-429e-8d07-0843a862ed8a service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:43.347499 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da3affc-e0ec-47fc-b0ad-7f10a07ba8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:43.348301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3affc-e0ec-47fc-b0ad-7f10a07ba8d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:43.348585 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3affc-e0ec-47fc-b0ad-7f10a07ba8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:43.348869 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3affc-e0ec-47fc-b0ad-7f10a07ba8d4 admin admin] PATH_INFO: `/v3/users/a51c40439edb4e2caf50da73865161f1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:43.349994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da3affc-e0ec-47fc-b0ad-7f10a07ba8d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a51c40439edb4e2caf50da73865161f1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:43.352875 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da3affc-e0ec-47fc-b0ad-7f10a07ba8d4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:43.552190 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:19:43.557823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8da3affc-e0ec-47fc-b0ad-7f10a07ba8d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a51c40439edb4e2caf50da73865161f1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:19:43.629835 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8da3affc-e0ec-47fc-b0ad-7f10a07ba8d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a51c40439edb4e2caf50da73865161f1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:19:43.638487 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8da3affc-e0ec-47fc-b0ad-7f10a07ba8d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a51c40439edb4e2caf50da73865161f1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:19:43.651571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8da3affc-e0ec-47fc-b0ad-7f10a07ba8d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a51c40439edb4e2caf50da73865161f1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:19:43.652356 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8da3affc-e0ec-47fc-b0ad-7f10a07ba8d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a51c40439edb4e2caf50da73865161f1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:19:43.658393 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1143/2267] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:19:43 2022] DELETE /identity/v3/users/a51c40439edb4e2caf50da73865161f1 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:19:43.662989 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90260b9a-f13f-4c58-a8d4-6f8cacbe83a6 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:43.663513 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90260b9a-f13f-4c58-a8d4-6f8cacbe83a6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:43.664183 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90260b9a-f13f-4c58-a8d4-6f8cacbe83a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:43.683266 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-90260b9a-f13f-4c58-a8d4-6f8cacbe83a6 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:19:43.684494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-90260b9a-f13f-4c58-a8d4-6f8cacbe83a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:43.754642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1125/2268] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:19:43.770162 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da3affc-e0ec-47fc-b0ad-7f10a07ba8d4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:43.771742 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8da3affc-e0ec-47fc-b0ad-7f10a07ba8d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:43.833244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da3affc-e0ec-47fc-b0ad-7f10a07ba8d4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:43.840044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59822cd-df2e-4caf-99e0-6b2ea96e96c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:43.840861 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59822cd-df2e-4caf-99e0-6b2ea96e96c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:43.841656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59822cd-df2e-4caf-99e0-6b2ea96e96c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:43.841939 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59822cd-df2e-4caf-99e0-6b2ea96e96c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:43.855308 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59822cd-df2e-4caf-99e0-6b2ea96e96c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:43.856860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59822cd-df2e-4caf-99e0-6b2ea96e96c9 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:43.869434 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1144/2269] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:19:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:19:44.358228 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90260b9a-f13f-4c58-a8d4-6f8cacbe83a6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:44.359719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-90260b9a-f13f-4c58-a8d4-6f8cacbe83a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:44.372106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90260b9a-f13f-4c58-a8d4-6f8cacbe83a6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:44.377989 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0ca6d-f12f-4431-8bb2-e983c67e4617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:44.378671 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0ca6d-f12f-4431-8bb2-e983c67e4617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:44.378952 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0ca6d-f12f-4431-8bb2-e983c67e4617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:44.379224 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0ca6d-f12f-4431-8bb2-e983c67e4617 admin admin] PATH_INFO: `/v3/projects/d5ad37fbf76a4d0da32e49247c5cf468` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:44.380400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d0ca6d-f12f-4431-8bb2-e983c67e4617 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5ad37fbf76a4d0da32e49247c5cf468)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:44.382919 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d0ca6d-f12f-4431-8bb2-e983c67e4617 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:44.483199 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-46d0ca6d-f12f-4431-8bb2-e983c67e4617 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5ad37fbf76a4d0da32e49247c5cf468', 'request_id': 'req-46d0ca6d-f12f-4431-8bb2-e983c67e4617'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:19:44.620909 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-46d0ca6d-f12f-4431-8bb2-e983c67e4617 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5ad37fbf76a4d0da32e49247c5cf468', 'request_id': 'req-46d0ca6d-f12f-4431-8bb2-e983c67e4617'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:19:44.628183 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-46d0ca6d-f12f-4431-8bb2-e983c67e4617 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5ad37fbf76a4d0da32e49247c5cf468', 'request_id': 'req-46d0ca6d-f12f-4431-8bb2-e983c67e4617'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:19:44.640531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-46d0ca6d-f12f-4431-8bb2-e983c67e4617 admin admin] The token cache is being invalidate because project d5ad37fbf76a4d0da32e49247c5cf468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:19:44.640823 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-46d0ca6d-f12f-4431-8bb2-e983c67e4617 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:19:44.641485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-46d0ca6d-f12f-4431-8bb2-e983c67e4617 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:19:44.643073 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1126/2270] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:19:44 2022] DELETE /identity/v3/projects/d5ad37fbf76a4d0da32e49247c5cf468 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:19:45.215751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5df6e7-ea50-4c5f-bb09-09eba58675f2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:45.216274 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5df6e7-ea50-4c5f-bb09-09eba58675f2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:45.216274 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5df6e7-ea50-4c5f-bb09-09eba58675f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:45.236838 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8e5df6e7-ea50-4c5f-bb09-09eba58675f2 None None] MFA Rules not processed for user `2f74a20789dc4ef697861146252bacba`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:19:45.238067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8e5df6e7-ea50-4c5f-bb09-09eba58675f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:45.300102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1145/2271] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:19:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:19:45.617546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0ca6d-f12f-4431-8bb2-e983c67e4617 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:45.619705 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-46d0ca6d-f12f-4431-8bb2-e983c67e4617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:45.647012 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:19:45.680325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0ca6d-f12f-4431-8bb2-e983c67e4617 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:45.684252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51570a-4e31-482e-8dc4-8c3ed484de36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:45.684888 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51570a-4e31-482e-8dc4-8c3ed484de36 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:45.685164 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51570a-4e31-482e-8dc4-8c3ed484de36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:45.685433 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51570a-4e31-482e-8dc4-8c3ed484de36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:45.687293 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3e51570a-4e31-482e-8dc4-8c3ed484de36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:45.697193 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51570a-4e31-482e-8dc4-8c3ed484de36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:45.698644 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51570a-4e31-482e-8dc4-8c3ed484de36 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:45.706822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1127/2272] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:19:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:19:45.866187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcbdda1-6a52-4a25-bf49-afecfb9f6141 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:45.866807 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcbdda1-6a52-4a25-bf49-afecfb9f6141 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:45.867117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcbdda1-6a52-4a25-bf49-afecfb9f6141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:45.886988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-fdcbdda1-6a52-4a25-bf49-afecfb9f6141 None None] MFA Rules not processed for user `5bf9229cd5d6429ea14249f8976e83cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:19:45.888205 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fdcbdda1-6a52-4a25-bf49-afecfb9f6141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:45.947555 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1146/2273] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:19:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:19:46.138985 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51570a-4e31-482e-8dc4-8c3ed484de36 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:46.140531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3e51570a-4e31-482e-8dc4-8c3ed484de36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:46.215128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51570a-4e31-482e-8dc4-8c3ed484de36 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:46.220472 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0968d5-7f71-4844-a82d-28fbc5809212 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cd39d9a583a4d73a5654017be106019', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:46.221079 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0968d5-7f71-4844-a82d-28fbc5809212 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:46.221367 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0968d5-7f71-4844-a82d-28fbc5809212 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:46.221638 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0968d5-7f71-4844-a82d-28fbc5809212 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:46.223741 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-db0968d5-7f71-4844-a82d-28fbc5809212 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:46.236261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0968d5-7f71-4844-a82d-28fbc5809212 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:46.237843 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0968d5-7f71-4844-a82d-28fbc5809212 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:46.248243 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1128/2274] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:19:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:19:46.292624 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcbdda1-6a52-4a25-bf49-afecfb9f6141 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:46.303505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcbdda1-6a52-4a25-bf49-afecfb9f6141 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:46.307742 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e5cb2-2eae-4664-9fce-c97fd555ca94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:46.308356 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e5cb2-2eae-4664-9fce-c97fd555ca94 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:46.308636 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e5cb2-2eae-4664-9fce-c97fd555ca94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:46.308901 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e5cb2-2eae-4664-9fce-c97fd555ca94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:46.310641 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-525e5cb2-2eae-4664-9fce-c97fd555ca94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:46.318010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:19:46.321065 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e5cb2-2eae-4664-9fce-c97fd555ca94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:46.322646 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e5cb2-2eae-4664-9fce-c97fd555ca94 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:46.337004 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1147/2275] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:19:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:19:50.455215 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0968d5-7f71-4844-a82d-28fbc5809212 service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:50.457928 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-db0968d5-7f71-4844-a82d-28fbc5809212 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:50.474905 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0968d5-7f71-4844-a82d-28fbc5809212 service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:50.477095 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687e5bf-53b7-482a-800b-18ad24a5e42b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:50.477595 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687e5bf-53b7-482a-800b-18ad24a5e42b service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:50.477940 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687e5bf-53b7-482a-800b-18ad24a5e42b service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:50.478843 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687e5bf-53b7-482a-800b-18ad24a5e42b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:50.480114 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2687e5bf-53b7-482a-800b-18ad24a5e42b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:50.497567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687e5bf-53b7-482a-800b-18ad24a5e42b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:50.499642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687e5bf-53b7-482a-800b-18ad24a5e42b service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:50.511163 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1129/2276] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:19:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:19:51.781417 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e5cb2-2eae-4664-9fce-c97fd555ca94 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:51.783507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-525e5cb2-2eae-4664-9fce-c97fd555ca94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:51.863688 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e5cb2-2eae-4664-9fce-c97fd555ca94 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:51.869322 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e7307-1638-447b-bf04-17494531a971 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:51.869935 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e7307-1638-447b-bf04-17494531a971 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:51.870212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e7307-1638-447b-bf04-17494531a971 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:51.870511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e7307-1638-447b-bf04-17494531a971 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:51.872243 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-939e7307-1638-447b-bf04-17494531a971 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:51.885644 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e7307-1638-447b-bf04-17494531a971 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:51.887260 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e7307-1638-447b-bf04-17494531a971 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:51.898704 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1148/2277] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:19:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:19:52.359540 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687e5bf-53b7-482a-800b-18ad24a5e42b service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:52.361148 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2687e5bf-53b7-482a-800b-18ad24a5e42b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:52.456548 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687e5bf-53b7-482a-800b-18ad24a5e42b service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:52.460362 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62539141-4e4c-42b5-9b87-ce0b9b5eab08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:52.461246 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62539141-4e4c-42b5-9b87-ce0b9b5eab08 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:52.461585 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62539141-4e4c-42b5-9b87-ce0b9b5eab08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:52.461928 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62539141-4e4c-42b5-9b87-ce0b9b5eab08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:52.463940 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-62539141-4e4c-42b5-9b87-ce0b9b5eab08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:52.533999 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:19:52.541433 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62539141-4e4c-42b5-9b87-ce0b9b5eab08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:52.543034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62539141-4e4c-42b5-9b87-ce0b9b5eab08 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:52.555670 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1130/2278] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:19:52 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:19:53.014970 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e7307-1638-447b-bf04-17494531a971 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:19:53.016425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-939e7307-1638-447b-bf04-17494531a971 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:53.030693 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e7307-1638-447b-bf04-17494531a971 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:19:53.036101 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822848b-d806-4420-908c-8ae76536163e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:19:53.036690 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822848b-d806-4420-908c-8ae76536163e service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:19:53.036968 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822848b-d806-4420-908c-8ae76536163e service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:19:53.037239 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822848b-d806-4420-908c-8ae76536163e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:19:53.039681 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c822848b-d806-4420-908c-8ae76536163e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:19:53.097029 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822848b-d806-4420-908c-8ae76536163e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:19:53.098639 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822848b-d806-4420-908c-8ae76536163e service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:19:53.108540 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1149/2279] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:19:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:20:02.663609 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:03.104158 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=64 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:05.696458 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62539141-4e4c-42b5-9b87-ce0b9b5eab08 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:05.718636 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62539141-4e4c-42b5-9b87-ce0b9b5eab08 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:05.725283 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9720910e-19be-4f9b-a353-d141df378719 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:05.729875 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720910e-19be-4f9b-a353-d141df378719 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:05.729875 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720910e-19be-4f9b-a353-d141df378719 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:05.729875 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720910e-19be-4f9b-a353-d141df378719 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:05.729875 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9720910e-19be-4f9b-a353-d141df378719 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:05.752298 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9720910e-19be-4f9b-a353-d141df378719 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:05.753989 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9720910e-19be-4f9b-a353-d141df378719 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:05.768113 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1131/2280] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:20:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:20:15.763067 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:21.114611 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822848b-d806-4420-908c-8ae76536163e service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:21.124127 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822848b-d806-4420-908c-8ae76536163e service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:21.127799 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41008307-f7d8-4829-94bd-2800fbe90718 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:21.128417 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41008307-f7d8-4829-94bd-2800fbe90718 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:21.128693 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41008307-f7d8-4829-94bd-2800fbe90718 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:21.128961 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41008307-f7d8-4829-94bd-2800fbe90718 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:21.130802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-41008307-f7d8-4829-94bd-2800fbe90718 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:21.194196 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41008307-f7d8-4829-94bd-2800fbe90718 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:21.195732 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41008307-f7d8-4829-94bd-2800fbe90718 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:21.207401 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1150/2281] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:20:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:20:22.836232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d030781-4c04-43fa-8c20-4a4ac25708f2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:22.836768 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d030781-4c04-43fa-8c20-4a4ac25708f2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:22.836864 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d030781-4c04-43fa-8c20-4a4ac25708f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:22.859003 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1d030781-4c04-43fa-8c20-4a4ac25708f2 None None] MFA Rules not processed for user `48feb2fc02f04e739c5e1bf7d68baa16`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:20:22.860159 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1d030781-4c04-43fa-8c20-4a4ac25708f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:22.868260 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1132/2282] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:20:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:20:25.752511 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41008307-f7d8-4829-94bd-2800fbe90718 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:25.754518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-41008307-f7d8-4829-94bd-2800fbe90718 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:25.767614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41008307-f7d8-4829-94bd-2800fbe90718 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:25.772601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbc9d7-80f4-4466-aac6-5a51c8351acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:25.782503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbc9d7-80f4-4466-aac6-5a51c8351acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:25.782503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbc9d7-80f4-4466-aac6-5a51c8351acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:25.782503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbc9d7-80f4-4466-aac6-5a51c8351acf admin admin] PATH_INFO: `/v3/users/7d2a65984c4f42fbb18977c7345c42ce` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:25.782503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bbc9d7-80f4-4466-aac6-5a51c8351acf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d2a65984c4f42fbb18977c7345c42ce)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:25.782503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bbc9d7-80f4-4466-aac6-5a51c8351acf admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:26.089460 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-64bbc9d7-80f4-4466-aac6-5a51c8351acf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d2a65984c4f42fbb18977c7345c42ce'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:26.240513 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-64bbc9d7-80f4-4466-aac6-5a51c8351acf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d2a65984c4f42fbb18977c7345c42ce'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:26.249021 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-64bbc9d7-80f4-4466-aac6-5a51c8351acf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d2a65984c4f42fbb18977c7345c42ce'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:26.261443 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-64bbc9d7-80f4-4466-aac6-5a51c8351acf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d2a65984c4f42fbb18977c7345c42ce'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:26.262083 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-64bbc9d7-80f4-4466-aac6-5a51c8351acf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d2a65984c4f42fbb18977c7345c42ce'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:26.267930 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1151/2283] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:20:25 2022] DELETE /identity/v3/users/7d2a65984c4f42fbb18977c7345c42ce => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:20:26.564905 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d030781-4c04-43fa-8c20-4a4ac25708f2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:26.576713 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d030781-4c04-43fa-8c20-4a4ac25708f2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:26.694901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca59f76c-f73e-4341-a4d7-4e83a488fde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:26.694901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59f76c-f73e-4341-a4d7-4e83a488fde2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:26.694901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59f76c-f73e-4341-a4d7-4e83a488fde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:26.694901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59f76c-f73e-4341-a4d7-4e83a488fde2 admin admin] PATH_INFO: `/v3/projects/2eb3ea42180f4a0ca5ca3c90962fb301` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:26.695613 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca59f76c-f73e-4341-a4d7-4e83a488fde2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eb3ea42180f4a0ca5ca3c90962fb301)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:26.695613 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca59f76c-f73e-4341-a4d7-4e83a488fde2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:26.917008 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ca59f76c-f73e-4341-a4d7-4e83a488fde2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eb3ea42180f4a0ca5ca3c90962fb301', 'request_id': 'req-ca59f76c-f73e-4341-a4d7-4e83a488fde2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:27.013578 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ca59f76c-f73e-4341-a4d7-4e83a488fde2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eb3ea42180f4a0ca5ca3c90962fb301', 'request_id': 'req-ca59f76c-f73e-4341-a4d7-4e83a488fde2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:27.021641 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ca59f76c-f73e-4341-a4d7-4e83a488fde2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eb3ea42180f4a0ca5ca3c90962fb301', 'request_id': 'req-ca59f76c-f73e-4341-a4d7-4e83a488fde2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:27.034691 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ca59f76c-f73e-4341-a4d7-4e83a488fde2 admin admin] The token cache is being invalidate because project 2eb3ea42180f4a0ca5ca3c90962fb301 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:20:27.034983 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ca59f76c-f73e-4341-a4d7-4e83a488fde2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:27.035646 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ca59f76c-f73e-4341-a4d7-4e83a488fde2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:27.037194 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1133/2284] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:20:26 2022] DELETE /identity/v3/projects/2eb3ea42180f4a0ca5ca3c90962fb301 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:20:27.043913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2728f-f996-4581-9856-ad96d3d6328e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:27.044445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2728f-f996-4581-9856-ad96d3d6328e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:27.044646 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2728f-f996-4581-9856-ad96d3d6328e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:27.065608 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c0a2728f-f996-4581-9856-ad96d3d6328e None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:20:27.066850 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c0a2728f-f996-4581-9856-ad96d3d6328e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:27.137572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1152/2285] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:20:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:20:27.145645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5274c2-d7e3-4cef-8a33-d46571f2ceaf None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:27.145927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5274c2-d7e3-4cef-8a33-d46571f2ceaf None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:27.146200 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5274c2-d7e3-4cef-8a33-d46571f2ceaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:27.181540 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6e5274c2-d7e3-4cef-8a33-d46571f2ceaf None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:20:27.182916 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6e5274c2-d7e3-4cef-8a33-d46571f2ceaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:27.192293 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1134/2286] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:20:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:20:27.210069 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2728f-f996-4581-9856-ad96d3d6328e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:27.221276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2728f-f996-4581-9856-ad96d3d6328e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:27.226139 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0bf35-619c-43a0-8fe3-4b36d16b41b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:27.226885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0bf35-619c-43a0-8fe3-4b36d16b41b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:27.227165 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0bf35-619c-43a0-8fe3-4b36d16b41b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:27.227437 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0bf35-619c-43a0-8fe3-4b36d16b41b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:27.228084 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe0bf35-619c-43a0-8fe3-4b36d16b41b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:20:27.228382 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe0bf35-619c-43a0-8fe3-4b36d16b41b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:27.230791 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe0bf35-619c-43a0-8fe3-4b36d16b41b6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:27.232490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:27.243111 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1153/2287] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:20:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 17:20:27.247885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5274c2-d7e3-4cef-8a33-d46571f2ceaf None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:27.257170 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5274c2-d7e3-4cef-8a33-d46571f2ceaf None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:27.262218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59134c06-17e6-4235-bc59-26232e4153eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:27.262846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59134c06-17e6-4235-bc59-26232e4153eb admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:27.263123 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59134c06-17e6-4235-bc59-26232e4153eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:27.263394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59134c06-17e6-4235-bc59-26232e4153eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:27.264055 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59134c06-17e6-4235-bc59-26232e4153eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:27.265969 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59134c06-17e6-4235-bc59-26232e4153eb admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:27.428747 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1135/2288] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:20:27 2022] POST /identity/v3/projects => generated 367 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:20:27.429990 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0bf35-619c-43a0-8fe3-4b36d16b41b6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:27.440439 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0bf35-619c-43a0-8fe3-4b36d16b41b6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:27.500744 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed06db-86b5-485d-9406-5fc988baba10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:27.501345 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed06db-86b5-485d-9406-5fc988baba10 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:27.501774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed06db-86b5-485d-9406-5fc988baba10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:27.501949 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed06db-86b5-485d-9406-5fc988baba10 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:27.502571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ed06db-86b5-485d-9406-5fc988baba10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:27.504464 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ed06db-86b5-485d-9406-5fc988baba10 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:27.623355 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1154/2289] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:20:27 2022] POST /identity/v3/users => generated 284 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:20:27.638884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59134c06-17e6-4235-bc59-26232e4153eb admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:27.638884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59134c06-17e6-4235-bc59-26232e4153eb admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:27.641491 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c0034-5b44-41c5-b31f-c281e334c934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:27.642201 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c0034-5b44-41c5-b31f-c281e334c934 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:27.642594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c0034-5b44-41c5-b31f-c281e334c934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:27.642918 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c0034-5b44-41c5-b31f-c281e334c934 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:27.643661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c0034-5b44-41c5-b31f-c281e334c934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:27.645337 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c0034-5b44-41c5-b31f-c281e334c934 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:27.658748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1136/2290] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:20:27.660001 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed06db-86b5-485d-9406-5fc988baba10 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:27.670917 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed06db-86b5-485d-9406-5fc988baba10 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:27.675708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724cb853-9a72-43ab-aa15-4094302b77f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:27.676320 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724cb853-9a72-43ab-aa15-4094302b77f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:27.676599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724cb853-9a72-43ab-aa15-4094302b77f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:27.676868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724cb853-9a72-43ab-aa15-4094302b77f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:27.677517 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724cb853-9a72-43ab-aa15-4094302b77f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:27.679228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724cb853-9a72-43ab-aa15-4094302b77f0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:27.694686 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1155/2291] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:20:27.695892 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c0034-5b44-41c5-b31f-c281e334c934 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:27.706658 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c0034-5b44-41c5-b31f-c281e334c934 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:27.711631 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45b0eb-2eb9-45cb-a954-0da0504a075b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:27.712301 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45b0eb-2eb9-45cb-a954-0da0504a075b admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:27.712577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45b0eb-2eb9-45cb-a954-0da0504a075b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:27.713126 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45b0eb-2eb9-45cb-a954-0da0504a075b admin admin] PATH_INFO: `/v3/projects/ba0d68ba24994f1cb1779307ad21b737/users/61f42315a0d64c12a0447c279bfdbc0d/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:27.729003 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e45b0eb-2eb9-45cb-a954-0da0504a075b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba0d68ba24994f1cb1779307ad21b737, user_id=61f42315a0d64c12a0447c279bfdbc0d, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:27.732339 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e45b0eb-2eb9-45cb-a954-0da0504a075b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:27.835435 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1137/2292] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:20:27 2022] PUT /identity/v3/projects/ba0d68ba24994f1cb1779307ad21b737/users/61f42315a0d64c12a0447c279bfdbc0d/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:20:27.847448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22b330-620e-49b5-9c46-3642954cf6ba None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:27.847762 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22b330-620e-49b5-9c46-3642954cf6ba None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:27.848049 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22b330-620e-49b5-9c46-3642954cf6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:27.879492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9e22b330-620e-49b5-9c46-3642954cf6ba None None] MFA Rules not processed for user `61f42315a0d64c12a0447c279bfdbc0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:20:27.880714 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9e22b330-620e-49b5-9c46-3642954cf6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:27.947730 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1156/2293] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:20:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:20:27.958001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45b0eb-2eb9-45cb-a954-0da0504a075b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:27.959799 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7e45b0eb-2eb9-45cb-a954-0da0504a075b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:27.983974 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:28.019258 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45b0eb-2eb9-45cb-a954-0da0504a075b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:28.023174 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c686241-43c4-43eb-95eb-a7ad7a54ad6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:28.023465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c686241-43c4-43eb-95eb-a7ad7a54ad6a service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:28.023738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c686241-43c4-43eb-95eb-a7ad7a54ad6a service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:28.024004 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c686241-43c4-43eb-95eb-a7ad7a54ad6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:28.032972 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c686241-43c4-43eb-95eb-a7ad7a54ad6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:28.034426 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c686241-43c4-43eb-95eb-a7ad7a54ad6a service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:28.044746 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1138/2294] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:20:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:20:35.069423 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22b330-620e-49b5-9c46-3642954cf6ba None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:35.071027 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9e22b330-620e-49b5-9c46-3642954cf6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:35.130543 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:35.145594 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22b330-620e-49b5-9c46-3642954cf6ba None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:35.151200 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4ddcc-6ad7-4d6a-b1c8-7bc4976f8b70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:35.153319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4ddcc-6ad7-4d6a-b1c8-7bc4976f8b70 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:35.153319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4ddcc-6ad7-4d6a-b1c8-7bc4976f8b70 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:35.153319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4ddcc-6ad7-4d6a-b1c8-7bc4976f8b70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:35.154236 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8fd4ddcc-6ad7-4d6a-b1c8-7bc4976f8b70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:35.166224 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4ddcc-6ad7-4d6a-b1c8-7bc4976f8b70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:35.167810 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4ddcc-6ad7-4d6a-b1c8-7bc4976f8b70 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:35.178437 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1157/2295] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:20:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:20:38.244951 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c686241-43c4-43eb-95eb-a7ad7a54ad6a service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:38.309022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c686241-43c4-43eb-95eb-a7ad7a54ad6a service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:38.314537 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d5fcf-3f1b-4096-b0d4-6891ba4a5900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:38.315173 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d5fcf-3f1b-4096-b0d4-6891ba4a5900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:38.315457 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d5fcf-3f1b-4096-b0d4-6891ba4a5900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:38.315727 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d5fcf-3f1b-4096-b0d4-6891ba4a5900 admin admin] PATH_INFO: `/v3/users/61f42315a0d64c12a0447c279bfdbc0d` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:38.316856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d5fcf-3f1b-4096-b0d4-6891ba4a5900 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61f42315a0d64c12a0447c279bfdbc0d)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:38.318852 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d5fcf-3f1b-4096-b0d4-6891ba4a5900 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:38.547830 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-889d5fcf-3f1b-4096-b0d4-6891ba4a5900 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61f42315a0d64c12a0447c279bfdbc0d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:38.668567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-889d5fcf-3f1b-4096-b0d4-6891ba4a5900 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61f42315a0d64c12a0447c279bfdbc0d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:38.682906 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-889d5fcf-3f1b-4096-b0d4-6891ba4a5900 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61f42315a0d64c12a0447c279bfdbc0d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:38.683623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-889d5fcf-3f1b-4096-b0d4-6891ba4a5900 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61f42315a0d64c12a0447c279bfdbc0d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:38.684305 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-889d5fcf-3f1b-4096-b0d4-6891ba4a5900 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61f42315a0d64c12a0447c279bfdbc0d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:38.698804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1139/2296] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:20:38 2022] DELETE /identity/v3/users/61f42315a0d64c12a0447c279bfdbc0d => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:20:38.704289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3eec82-7ff2-4064-b7db-9d56d7f2a20f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:38.704804 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3eec82-7ff2-4064-b7db-9d56d7f2a20f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:38.705065 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3eec82-7ff2-4064-b7db-9d56d7f2a20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:38.725161 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5f3eec82-7ff2-4064-b7db-9d56d7f2a20f None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:20:38.726510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5f3eec82-7ff2-4064-b7db-9d56d7f2a20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:38.790415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1158/2297] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:20:38.800999 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d5fcf-3f1b-4096-b0d4-6891ba4a5900 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:38.802415 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-889d5fcf-3f1b-4096-b0d4-6891ba4a5900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:38.849326 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:38.861761 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d5fcf-3f1b-4096-b0d4-6891ba4a5900 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:38.865876 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a92e2-bd14-43be-9e2f-28142c3ad979 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:38.866533 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a92e2-bd14-43be-9e2f-28142c3ad979 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:38.866856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a92e2-bd14-43be-9e2f-28142c3ad979 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:38.867140 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a92e2-bd14-43be-9e2f-28142c3ad979 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:38.878006 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a92e2-bd14-43be-9e2f-28142c3ad979 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:38.879644 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a92e2-bd14-43be-9e2f-28142c3ad979 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:38.893562 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1140/2298] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:20:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:20:39.523517 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3eec82-7ff2-4064-b7db-9d56d7f2a20f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:39.525060 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5f3eec82-7ff2-4064-b7db-9d56d7f2a20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:39.550050 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3eec82-7ff2-4064-b7db-9d56d7f2a20f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:39.555075 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c618bb-6a08-4e97-8d63-8d747428c28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:39.555745 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c618bb-6a08-4e97-8d63-8d747428c28b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:39.556414 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c618bb-6a08-4e97-8d63-8d747428c28b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:39.556414 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c618bb-6a08-4e97-8d63-8d747428c28b admin admin] PATH_INFO: `/v3/projects/ba0d68ba24994f1cb1779307ad21b737` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:39.558206 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c618bb-6a08-4e97-8d63-8d747428c28b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba0d68ba24994f1cb1779307ad21b737)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:39.560724 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c618bb-6a08-4e97-8d63-8d747428c28b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:39.713063 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-39c618bb-6a08-4e97-8d63-8d747428c28b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba0d68ba24994f1cb1779307ad21b737', 'request_id': 'req-39c618bb-6a08-4e97-8d63-8d747428c28b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:39.720866 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-39c618bb-6a08-4e97-8d63-8d747428c28b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba0d68ba24994f1cb1779307ad21b737', 'request_id': 'req-39c618bb-6a08-4e97-8d63-8d747428c28b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:39.811629 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-39c618bb-6a08-4e97-8d63-8d747428c28b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba0d68ba24994f1cb1779307ad21b737', 'request_id': 'req-39c618bb-6a08-4e97-8d63-8d747428c28b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:39.824952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-39c618bb-6a08-4e97-8d63-8d747428c28b admin admin] The token cache is being invalidate because project ba0d68ba24994f1cb1779307ad21b737 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:20:39.825271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-39c618bb-6a08-4e97-8d63-8d747428c28b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:39.825940 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-39c618bb-6a08-4e97-8d63-8d747428c28b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:39.827481 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1159/2299] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:20:39 2022] DELETE /identity/v3/projects/ba0d68ba24994f1cb1779307ad21b737 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:20:39.832691 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eaccc9-fd61-4e8f-ad13-e248574301a9 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:39.833224 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eaccc9-fd61-4e8f-ad13-e248574301a9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:39.833224 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eaccc9-fd61-4e8f-ad13-e248574301a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:39.855844 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-31eaccc9-fd61-4e8f-ad13-e248574301a9 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:20:39.856545 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-31eaccc9-fd61-4e8f-ad13-e248574301a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:39.921044 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1141/2300] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:20:39.929828 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835e0e4-0a04-471e-a63a-b4cc8f112ee5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:39.930167 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835e0e4-0a04-471e-a63a-b4cc8f112ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:39.930432 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835e0e4-0a04-471e-a63a-b4cc8f112ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:39.950657 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4835e0e4-0a04-471e-a63a-b4cc8f112ee5 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:20:39.951883 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4835e0e4-0a04-471e-a63a-b4cc8f112ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:39.960505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1160/2301] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:20:39.965033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eaccc9-fd61-4e8f-ad13-e248574301a9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:39.975902 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eaccc9-fd61-4e8f-ad13-e248574301a9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:39.980718 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f580d320-29c0-4bdb-8325-722c89f2e4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:39.981321 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580d320-29c0-4bdb-8325-722c89f2e4c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:39.981601 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580d320-29c0-4bdb-8325-722c89f2e4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:39.981867 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580d320-29c0-4bdb-8325-722c89f2e4c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:39.982667 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f580d320-29c0-4bdb-8325-722c89f2e4c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:20:39.982966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f580d320-29c0-4bdb-8325-722c89f2e4c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:39.985292 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f580d320-29c0-4bdb-8325-722c89f2e4c6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:39.999866 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1142/2302] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:20:40.007744 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835e0e4-0a04-471e-a63a-b4cc8f112ee5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:40.017675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835e0e4-0a04-471e-a63a-b4cc8f112ee5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:40.023069 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5711e-b987-4329-883b-6259dd4c5eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:40.023681 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5711e-b987-4329-883b-6259dd4c5eee admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:40.023998 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5711e-b987-4329-883b-6259dd4c5eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:40.024297 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5711e-b987-4329-883b-6259dd4c5eee admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:40.024922 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f5711e-b987-4329-883b-6259dd4c5eee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:40.026888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f5711e-b987-4329-883b-6259dd4c5eee admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:40.110771 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1161/2303] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:20:40 2022] POST /identity/v3/projects => generated 403 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:20:40.115085 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f580d320-29c0-4bdb-8325-722c89f2e4c6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:40.126983 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f580d320-29c0-4bdb-8325-722c89f2e4c6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:40.182691 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9190b-36af-4ab0-9835-25af91e55fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:40.183292 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9190b-36af-4ab0-9835-25af91e55fbb admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:40.183570 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9190b-36af-4ab0-9835-25af91e55fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:40.183846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9190b-36af-4ab0-9835-25af91e55fbb admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:40.184506 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9190b-36af-4ab0-9835-25af91e55fbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:40.186432 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9190b-36af-4ab0-9835-25af91e55fbb admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:40.409396 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1143/2304] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:20:40 2022] POST /identity/v3/users => generated 302 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:20:40.410442 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5711e-b987-4329-883b-6259dd4c5eee admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:40.421310 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5711e-b987-4329-883b-6259dd4c5eee admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:40.426334 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8730c-ca50-41ad-a1e7-12d7004b0eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:40.426999 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8730c-ca50-41ad-a1e7-12d7004b0eaa admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:40.427284 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8730c-ca50-41ad-a1e7-12d7004b0eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:40.427548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8730c-ca50-41ad-a1e7-12d7004b0eaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:40.428222 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8730c-ca50-41ad-a1e7-12d7004b0eaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:40.429816 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8730c-ca50-41ad-a1e7-12d7004b0eaa admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:40.441214 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1162/2305] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:20:40.445487 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9190b-36af-4ab0-9835-25af91e55fbb admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:40.456372 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9190b-36af-4ab0-9835-25af91e55fbb admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:40.461065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2efb19-c1e5-4d31-8a11-a176ec41e12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:40.461663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2efb19-c1e5-4d31-8a11-a176ec41e12a admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:40.461939 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2efb19-c1e5-4d31-8a11-a176ec41e12a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:40.462205 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2efb19-c1e5-4d31-8a11-a176ec41e12a admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:40.462899 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2efb19-c1e5-4d31-8a11-a176ec41e12a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:40.464489 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2efb19-c1e5-4d31-8a11-a176ec41e12a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:40.479547 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1144/2306] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:20:40.480706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8730c-ca50-41ad-a1e7-12d7004b0eaa admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:40.493509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8730c-ca50-41ad-a1e7-12d7004b0eaa admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:40.500037 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e15f5-0c4d-40dd-9728-4db5eb79b65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:40.503117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e15f5-0c4d-40dd-9728-4db5eb79b65f admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:40.503117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e15f5-0c4d-40dd-9728-4db5eb79b65f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:40.503117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e15f5-0c4d-40dd-9728-4db5eb79b65f admin admin] PATH_INFO: `/v3/projects/e8a7f67181ba4eb087cfbbcb84e7e9ba/users/b02ec08e01194da085f4339bb106baa6/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:40.519001 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e15f5-0c4d-40dd-9728-4db5eb79b65f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8a7f67181ba4eb087cfbbcb84e7e9ba, user_id=b02ec08e01194da085f4339bb106baa6, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:40.521741 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e15f5-0c4d-40dd-9728-4db5eb79b65f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:40.623786 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1163/2307] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:20:40 2022] PUT /identity/v3/projects/e8a7f67181ba4eb087cfbbcb84e7e9ba/users/b02ec08e01194da085f4339bb106baa6/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:20:40.635144 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507eff0-9c0a-4577-b693-42b4b25005bc None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:40.635485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507eff0-9c0a-4577-b693-42b4b25005bc None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:40.635756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507eff0-9c0a-4577-b693-42b4b25005bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:40.663172 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3507eff0-9c0a-4577-b693-42b4b25005bc None None] MFA Rules not processed for user `b02ec08e01194da085f4339bb106baa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:20:40.664364 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3507eff0-9c0a-4577-b693-42b4b25005bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:40.703433 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:40.728695 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1145/2308] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:20:40 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:20:40.733152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e15f5-0c4d-40dd-9728-4db5eb79b65f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:40.748316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:40.799280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e15f5-0c4d-40dd-9728-4db5eb79b65f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:40.804067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ead8c6-4db5-4dbe-9673-30203c8cdf6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:40.804662 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ead8c6-4db5-4dbe-9673-30203c8cdf6c admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:40.804940 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ead8c6-4db5-4dbe-9673-30203c8cdf6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:40.805211 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ead8c6-4db5-4dbe-9673-30203c8cdf6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:40.805829 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ead8c6-4db5-4dbe-9673-30203c8cdf6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:40.807790 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ead8c6-4db5-4dbe-9673-30203c8cdf6c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:40.973506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1164/2309] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:20:40 2022] POST /identity/v3/projects => generated 403 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:20:40.977613 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507eff0-9c0a-4577-b693-42b4b25005bc None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:40.988317 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507eff0-9c0a-4577-b693-42b4b25005bc None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:41.046003 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4e563-bc6e-4935-93dc-25e07ab2cdf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:41.047227 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4e563-bc6e-4935-93dc-25e07ab2cdf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:41.047503 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4e563-bc6e-4935-93dc-25e07ab2cdf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:41.047770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4e563-bc6e-4935-93dc-25e07ab2cdf3 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:41.048407 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4e563-bc6e-4935-93dc-25e07ab2cdf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:41.050316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4e563-bc6e-4935-93dc-25e07ab2cdf3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:41.195838 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1146/2310] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:20:40 2022] POST /identity/v3/users => generated 308 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:20:41.199738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ead8c6-4db5-4dbe-9673-30203c8cdf6c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:41.210042 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ead8c6-4db5-4dbe-9673-30203c8cdf6c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:41.214620 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e60d26-3927-4abd-95cc-699817e38b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:41.215229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e60d26-3927-4abd-95cc-699817e38b58 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:41.215493 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e60d26-3927-4abd-95cc-699817e38b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:41.215758 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e60d26-3927-4abd-95cc-699817e38b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:41.216400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e60d26-3927-4abd-95cc-699817e38b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:41.218013 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e60d26-3927-4abd-95cc-699817e38b58 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:41.230817 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1165/2311] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:20:41.236702 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4e563-bc6e-4935-93dc-25e07ab2cdf3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:41.247640 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4e563-bc6e-4935-93dc-25e07ab2cdf3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:41.252647 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b99cea-5a6b-4b95-822d-93830977d23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:41.253298 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b99cea-5a6b-4b95-822d-93830977d23b admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:41.253577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b99cea-5a6b-4b95-822d-93830977d23b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:41.253851 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b99cea-5a6b-4b95-822d-93830977d23b admin admin] PATH_INFO: `/v3/projects/233fed9ebf0a4e07ab61a7445bdebcfa/users/34c099eebc3e4dd28717c52e5e80e78a/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:41.269211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b99cea-5a6b-4b95-822d-93830977d23b admin admin] RBAC: Authorizing `identity:create_grant(project_id=233fed9ebf0a4e07ab61a7445bdebcfa, user_id=34c099eebc3e4dd28717c52e5e80e78a, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:41.271930 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b99cea-5a6b-4b95-822d-93830977d23b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:41.395176 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1147/2312] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:20:41 2022] PUT /identity/v3/projects/233fed9ebf0a4e07ab61a7445bdebcfa/users/34c099eebc3e4dd28717c52e5e80e78a/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:20:41.399213 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e60d26-3927-4abd-95cc-699817e38b58 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:41.409540 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e60d26-3927-4abd-95cc-699817e38b58 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:41.467896 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd0aeec-b874-410b-b741-a2a47c7417aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:41.468573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0aeec-b874-410b-b741-a2a47c7417aa admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:41.468820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0aeec-b874-410b-b741-a2a47c7417aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:41.469092 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0aeec-b874-410b-b741-a2a47c7417aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:41.469769 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd0aeec-b874-410b-b741-a2a47c7417aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:41.471440 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd0aeec-b874-410b-b741-a2a47c7417aa admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:41.481477 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1166/2313] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:20:41.485911 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b99cea-5a6b-4b95-822d-93830977d23b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:41.496556 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b99cea-5a6b-4b95-822d-93830977d23b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:41.501784 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842c19a-5447-42a3-8234-aa33632a92ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:41.502466 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842c19a-5447-42a3-8234-aa33632a92ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:41.502747 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842c19a-5447-42a3-8234-aa33632a92ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:41.503017 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842c19a-5447-42a3-8234-aa33632a92ee admin admin] PATH_INFO: `/v3/projects/233fed9ebf0a4e07ab61a7445bdebcfa/users/34c099eebc3e4dd28717c52e5e80e78a/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:41.504948 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842c19a-5447-42a3-8234-aa33632a92ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=233fed9ebf0a4e07ab61a7445bdebcfa, user_id=34c099eebc3e4dd28717c52e5e80e78a, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:41.507904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842c19a-5447-42a3-8234-aa33632a92ee admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:41.518757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1148/2314] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:20:41 2022] PUT /identity/v3/projects/233fed9ebf0a4e07ab61a7445bdebcfa/users/34c099eebc3e4dd28717c52e5e80e78a/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:20:41.530451 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e184171-762f-44c3-a9aa-b76ee114b037 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:41.530720 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e184171-762f-44c3-a9aa-b76ee114b037 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:41.531505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e184171-762f-44c3-a9aa-b76ee114b037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:41.559174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7e184171-762f-44c3-a9aa-b76ee114b037 None None] MFA Rules not processed for user `34c099eebc3e4dd28717c52e5e80e78a`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:20:41.560372 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7e184171-762f-44c3-a9aa-b76ee114b037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:41.629278 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1167/2315] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:20:41 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:20:41.641278 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842c19a-5447-42a3-8234-aa33632a92ee admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:41.642972 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f842c19a-5447-42a3-8234-aa33632a92ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:41.702330 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842c19a-5447-42a3-8234-aa33632a92ee admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:41.707152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb419bc-3e73-43af-8509-ca8931a0fe02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:41.707762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb419bc-3e73-43af-8509-ca8931a0fe02 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:41.708039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb419bc-3e73-43af-8509-ca8931a0fe02 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:41.708314 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb419bc-3e73-43af-8509-ca8931a0fe02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:41.717563 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb419bc-3e73-43af-8509-ca8931a0fe02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:41.719771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb419bc-3e73-43af-8509-ca8931a0fe02 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:41.729160 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1149/2316] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:20:41 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:20:41.887508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e184171-762f-44c3-a9aa-b76ee114b037 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:41.892771 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:41.955504 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e184171-762f-44c3-a9aa-b76ee114b037 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:41.960259 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ed24e-a4ee-46a9-a1fd-4582d27882a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:41.961139 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ed24e-a4ee-46a9-a1fd-4582d27882a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:41.961415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ed24e-a4ee-46a9-a1fd-4582d27882a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:41.961685 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ed24e-a4ee-46a9-a1fd-4582d27882a0 admin admin] PATH_INFO: `/v3/users/b02ec08e01194da085f4339bb106baa6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:41.962850 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ed24e-a4ee-46a9-a1fd-4582d27882a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b02ec08e01194da085f4339bb106baa6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:41.965005 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ed24e-a4ee-46a9-a1fd-4582d27882a0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:42.186423 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-157ed24e-a4ee-46a9-a1fd-4582d27882a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b02ec08e01194da085f4339bb106baa6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:42.252219 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-157ed24e-a4ee-46a9-a1fd-4582d27882a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b02ec08e01194da085f4339bb106baa6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:42.261305 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-157ed24e-a4ee-46a9-a1fd-4582d27882a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b02ec08e01194da085f4339bb106baa6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:42.275043 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-157ed24e-a4ee-46a9-a1fd-4582d27882a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b02ec08e01194da085f4339bb106baa6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:42.275785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-157ed24e-a4ee-46a9-a1fd-4582d27882a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b02ec08e01194da085f4339bb106baa6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:42.282002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1168/2317] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:20:41 2022] DELETE /identity/v3/users/b02ec08e01194da085f4339bb106baa6 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:20:42.286871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8607f9-ae1d-4d7f-8849-96905d703ae1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:42.287354 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8607f9-ae1d-4d7f-8849-96905d703ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:42.287465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8607f9-ae1d-4d7f-8849-96905d703ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:42.306796 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-da8607f9-ae1d-4d7f-8849-96905d703ae1 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:20:42.307915 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-da8607f9-ae1d-4d7f-8849-96905d703ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:42.340694 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:42.370397 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1150/2318] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:20:42.381249 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ed24e-a4ee-46a9-a1fd-4582d27882a0 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:42.382687 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-157ed24e-a4ee-46a9-a1fd-4582d27882a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:42.449037 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ed24e-a4ee-46a9-a1fd-4582d27882a0 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:42.453790 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771b198-05a8-4cd0-bb00-f3440966bd22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:42.455043 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771b198-05a8-4cd0-bb00-f3440966bd22 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:42.455525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771b198-05a8-4cd0-bb00-f3440966bd22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:42.455971 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771b198-05a8-4cd0-bb00-f3440966bd22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:42.472612 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771b198-05a8-4cd0-bb00-f3440966bd22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:42.474466 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771b198-05a8-4cd0-bb00-f3440966bd22 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:42.492925 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1169/2319] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:20:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:20:42.967966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8607f9-ae1d-4d7f-8849-96905d703ae1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:42.970143 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-da8607f9-ae1d-4d7f-8849-96905d703ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:42.982478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8607f9-ae1d-4d7f-8849-96905d703ae1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:42.988866 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073a5f5-cfc6-4dc9-8d0e-8d2afe7b4a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:42.989668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073a5f5-cfc6-4dc9-8d0e-8d2afe7b4a03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:42.989947 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073a5f5-cfc6-4dc9-8d0e-8d2afe7b4a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:42.990215 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073a5f5-cfc6-4dc9-8d0e-8d2afe7b4a03 admin admin] PATH_INFO: `/v3/projects/e8a7f67181ba4eb087cfbbcb84e7e9ba` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:42.991370 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3073a5f5-cfc6-4dc9-8d0e-8d2afe7b4a03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8a7f67181ba4eb087cfbbcb84e7e9ba)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:42.993365 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3073a5f5-cfc6-4dc9-8d0e-8d2afe7b4a03 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:43.143690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3073a5f5-cfc6-4dc9-8d0e-8d2afe7b4a03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8a7f67181ba4eb087cfbbcb84e7e9ba', 'request_id': 'req-3073a5f5-cfc6-4dc9-8d0e-8d2afe7b4a03'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:43.263060 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3073a5f5-cfc6-4dc9-8d0e-8d2afe7b4a03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8a7f67181ba4eb087cfbbcb84e7e9ba', 'request_id': 'req-3073a5f5-cfc6-4dc9-8d0e-8d2afe7b4a03'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:43.270008 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3073a5f5-cfc6-4dc9-8d0e-8d2afe7b4a03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8a7f67181ba4eb087cfbbcb84e7e9ba', 'request_id': 'req-3073a5f5-cfc6-4dc9-8d0e-8d2afe7b4a03'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:43.280597 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3073a5f5-cfc6-4dc9-8d0e-8d2afe7b4a03 admin admin] The token cache is being invalidate because project e8a7f67181ba4eb087cfbbcb84e7e9ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:20:43.280961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3073a5f5-cfc6-4dc9-8d0e-8d2afe7b4a03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:43.281613 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3073a5f5-cfc6-4dc9-8d0e-8d2afe7b4a03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:43.283155 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1151/2320] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:20:42 2022] DELETE /identity/v3/projects/e8a7f67181ba4eb087cfbbcb84e7e9ba => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:20:43.287116 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771b198-05a8-4cd0-bb00-f3440966bd22 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:43.297538 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771b198-05a8-4cd0-bb00-f3440966bd22 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:43.317741 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:43.353031 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142cd138-6151-40ce-b113-823b957bbe4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:43.353652 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cd138-6151-40ce-b113-823b957bbe4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:43.353927 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cd138-6151-40ce-b113-823b957bbe4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:43.354194 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cd138-6151-40ce-b113-823b957bbe4f admin admin] PATH_INFO: `/v3/users/34c099eebc3e4dd28717c52e5e80e78a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:43.355340 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142cd138-6151-40ce-b113-823b957bbe4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=34c099eebc3e4dd28717c52e5e80e78a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:43.357256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142cd138-6151-40ce-b113-823b957bbe4f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:43.580275 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-142cd138-6151-40ce-b113-823b957bbe4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34c099eebc3e4dd28717c52e5e80e78a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:43.720364 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-142cd138-6151-40ce-b113-823b957bbe4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34c099eebc3e4dd28717c52e5e80e78a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:43.728216 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-142cd138-6151-40ce-b113-823b957bbe4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34c099eebc3e4dd28717c52e5e80e78a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:43.740912 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-142cd138-6151-40ce-b113-823b957bbe4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34c099eebc3e4dd28717c52e5e80e78a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:43.741623 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-142cd138-6151-40ce-b113-823b957bbe4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34c099eebc3e4dd28717c52e5e80e78a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:43.747416 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1170/2321] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:20:43 2022] DELETE /identity/v3/users/34c099eebc3e4dd28717c52e5e80e78a => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:20:44.316591 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073a5f5-cfc6-4dc9-8d0e-8d2afe7b4a03 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:44.318214 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3073a5f5-cfc6-4dc9-8d0e-8d2afe7b4a03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:44.334418 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073a5f5-cfc6-4dc9-8d0e-8d2afe7b4a03 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:44.393981 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3bf53b-8e47-4615-9698-04e9dace803e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:44.394637 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bf53b-8e47-4615-9698-04e9dace803e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:44.394916 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bf53b-8e47-4615-9698-04e9dace803e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:44.395190 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bf53b-8e47-4615-9698-04e9dace803e admin admin] PATH_INFO: `/v3/projects/233fed9ebf0a4e07ab61a7445bdebcfa` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:44.396311 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3bf53b-8e47-4615-9698-04e9dace803e admin admin] RBAC: Authorizing `identity:delete_project(project_id=233fed9ebf0a4e07ab61a7445bdebcfa)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:44.398212 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3bf53b-8e47-4615-9698-04e9dace803e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:44.516546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9f3bf53b-8e47-4615-9698-04e9dace803e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '233fed9ebf0a4e07ab61a7445bdebcfa', 'request_id': 'req-9f3bf53b-8e47-4615-9698-04e9dace803e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:44.612664 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9f3bf53b-8e47-4615-9698-04e9dace803e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '233fed9ebf0a4e07ab61a7445bdebcfa', 'request_id': 'req-9f3bf53b-8e47-4615-9698-04e9dace803e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:44.620038 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9f3bf53b-8e47-4615-9698-04e9dace803e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '233fed9ebf0a4e07ab61a7445bdebcfa', 'request_id': 'req-9f3bf53b-8e47-4615-9698-04e9dace803e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:44.621493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:44.632998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9f3bf53b-8e47-4615-9698-04e9dace803e admin admin] The token cache is being invalidate because project 233fed9ebf0a4e07ab61a7445bdebcfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:20:44.634876 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9f3bf53b-8e47-4615-9698-04e9dace803e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:44.634876 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9f3bf53b-8e47-4615-9698-04e9dace803e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:44.636270 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1152/2322] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:20:44 2022] DELETE /identity/v3/projects/233fed9ebf0a4e07ab61a7445bdebcfa => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:20:44.641717 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a203b-6757-45d1-a638-09f59ba32849 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:44.642209 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a203b-6757-45d1-a638-09f59ba32849 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:44.642325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a203b-6757-45d1-a638-09f59ba32849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:44.661867 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0a7a203b-6757-45d1-a638-09f59ba32849 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:20:44.663111 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0a7a203b-6757-45d1-a638-09f59ba32849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:44.723749 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1171/2323] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:20:44.731704 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248165e-4508-43f5-b119-543be683351c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:44.731984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248165e-4508-43f5-b119-543be683351c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:44.732252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248165e-4508-43f5-b119-543be683351c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:44.751968 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5248165e-4508-43f5-b119-543be683351c None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:20:44.753174 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5248165e-4508-43f5-b119-543be683351c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:44.761455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1153/2324] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:20:44.768596 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a203b-6757-45d1-a638-09f59ba32849 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:44.778334 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a203b-6757-45d1-a638-09f59ba32849 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:44.783814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab69ff8-bc5a-4c9d-81a5-8dc8edb3d2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:44.784404 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab69ff8-bc5a-4c9d-81a5-8dc8edb3d2f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:44.784683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab69ff8-bc5a-4c9d-81a5-8dc8edb3d2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:44.784956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab69ff8-bc5a-4c9d-81a5-8dc8edb3d2f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:44.785598 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab69ff8-bc5a-4c9d-81a5-8dc8edb3d2f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:20:44.785943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab69ff8-bc5a-4c9d-81a5-8dc8edb3d2f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:44.787617 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab69ff8-bc5a-4c9d-81a5-8dc8edb3d2f1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:44.799445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1172/2325] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:20:44.805210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248165e-4508-43f5-b119-543be683351c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:44.818256 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248165e-4508-43f5-b119-543be683351c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:44.823884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0148803d-23f6-4795-a334-5d7ab8b94975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:44.824478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148803d-23f6-4795-a334-5d7ab8b94975 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:44.824755 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148803d-23f6-4795-a334-5d7ab8b94975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:44.825022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148803d-23f6-4795-a334-5d7ab8b94975 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:44.825658 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0148803d-23f6-4795-a334-5d7ab8b94975 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:44.828204 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0148803d-23f6-4795-a334-5d7ab8b94975 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:44.945478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1154/2326] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:20:44 2022] POST /identity/v3/projects => generated 365 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:20:44.949316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab69ff8-bc5a-4c9d-81a5-8dc8edb3d2f1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:44.960271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab69ff8-bc5a-4c9d-81a5-8dc8edb3d2f1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:44.991703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:45.021328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41abad60-0fa2-4b3b-946e-3788b33cba74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:45.021961 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41abad60-0fa2-4b3b-946e-3788b33cba74 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:45.022235 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41abad60-0fa2-4b3b-946e-3788b33cba74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:45.022544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41abad60-0fa2-4b3b-946e-3788b33cba74 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:45.023169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41abad60-0fa2-4b3b-946e-3788b33cba74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:45.025096 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41abad60-0fa2-4b3b-946e-3788b33cba74 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:45.164726 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1173/2327] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:20:44 2022] POST /identity/v3/users => generated 283 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:20:45.165919 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0148803d-23f6-4795-a334-5d7ab8b94975 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:45.177926 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0148803d-23f6-4795-a334-5d7ab8b94975 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:45.182592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd888b45-4155-4d2b-aa45-6c7d514bf1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:45.183202 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd888b45-4155-4d2b-aa45-6c7d514bf1cd admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:45.183480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd888b45-4155-4d2b-aa45-6c7d514bf1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:45.183751 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd888b45-4155-4d2b-aa45-6c7d514bf1cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:45.184398 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd888b45-4155-4d2b-aa45-6c7d514bf1cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:45.186013 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd888b45-4155-4d2b-aa45-6c7d514bf1cd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:45.198485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1155/2328] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:20:45.208639 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41abad60-0fa2-4b3b-946e-3788b33cba74 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:45.219449 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41abad60-0fa2-4b3b-946e-3788b33cba74 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:45.224524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd28f2-5536-4817-9850-eef29d6281c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:45.225115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd28f2-5536-4817-9850-eef29d6281c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:45.225389 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd28f2-5536-4817-9850-eef29d6281c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:45.225657 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd28f2-5536-4817-9850-eef29d6281c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:45.226290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd28f2-5536-4817-9850-eef29d6281c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:45.227941 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd28f2-5536-4817-9850-eef29d6281c3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:45.240217 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1174/2329] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:20:45.245454 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd888b45-4155-4d2b-aa45-6c7d514bf1cd admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:45.256022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd888b45-4155-4d2b-aa45-6c7d514bf1cd admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:45.260797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8429a9c-44af-4571-9320-149543ad6107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:45.261432 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8429a9c-44af-4571-9320-149543ad6107 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:45.261708 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8429a9c-44af-4571-9320-149543ad6107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:45.261977 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8429a9c-44af-4571-9320-149543ad6107 admin admin] PATH_INFO: `/v3/projects/b3ce06fb3a0e4696b4a0ffed8859deca/users/0d6c690821b74bcc99a6fb32cc59aa11/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:45.277242 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8429a9c-44af-4571-9320-149543ad6107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3ce06fb3a0e4696b4a0ffed8859deca, user_id=0d6c690821b74bcc99a6fb32cc59aa11, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:45.280103 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8429a9c-44af-4571-9320-149543ad6107 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:45.367928 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1156/2330] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:20:45 2022] PUT /identity/v3/projects/b3ce06fb3a0e4696b4a0ffed8859deca/users/0d6c690821b74bcc99a6fb32cc59aa11/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:20:45.380504 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d87b3-b41e-4e7d-a01c-0e93bdc61eaf None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:45.380842 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d87b3-b41e-4e7d-a01c-0e93bdc61eaf None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:45.381633 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d87b3-b41e-4e7d-a01c-0e93bdc61eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:45.409830 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e89d87b3-b41e-4e7d-a01c-0e93bdc61eaf None None] MFA Rules not processed for user `0d6c690821b74bcc99a6fb32cc59aa11`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:20:45.411174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e89d87b3-b41e-4e7d-a01c-0e93bdc61eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:45.475001 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1175/2331] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:20:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:20:45.479561 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8429a9c-44af-4571-9320-149543ad6107 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:45.542231 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8429a9c-44af-4571-9320-149543ad6107 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:45.547626 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc8a00-9188-4171-93b3-9f031184b1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:45.548447 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc8a00-9188-4171-93b3-9f031184b1c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:45.548796 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc8a00-9188-4171-93b3-9f031184b1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:45.549115 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc8a00-9188-4171-93b3-9f031184b1c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:45.549790 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc8a00-9188-4171-93b3-9f031184b1c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:45.551822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc8a00-9188-4171-93b3-9f031184b1c0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:45.686570 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1157/2332] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:20:45 2022] POST /identity/v3/projects => generated 363 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:20:45.691409 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d87b3-b41e-4e7d-a01c-0e93bdc61eaf None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:45.702115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d87b3-b41e-4e7d-a01c-0e93bdc61eaf None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:45.759488 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b41197-137c-448a-b145-6b82cbf4ae8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:45.760070 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b41197-137c-448a-b145-6b82cbf4ae8c admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:45.760352 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b41197-137c-448a-b145-6b82cbf4ae8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:45.760625 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b41197-137c-448a-b145-6b82cbf4ae8c admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:45.761239 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b41197-137c-448a-b145-6b82cbf4ae8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:45.763179 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b41197-137c-448a-b145-6b82cbf4ae8c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:45.868551 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1176/2333] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:20:45 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:20:45.873393 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc8a00-9188-4171-93b3-9f031184b1c0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:45.884822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc8a00-9188-4171-93b3-9f031184b1c0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:45.889490 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6eadae-0844-4fa5-82c7-73f60f6c2d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:45.890752 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6eadae-0844-4fa5-82c7-73f60f6c2d22 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:45.891349 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6eadae-0844-4fa5-82c7-73f60f6c2d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:45.891638 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6eadae-0844-4fa5-82c7-73f60f6c2d22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:45.892278 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6eadae-0844-4fa5-82c7-73f60f6c2d22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:45.893874 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6eadae-0844-4fa5-82c7-73f60f6c2d22 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:45.905470 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1158/2334] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:20:45.910435 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b41197-137c-448a-b145-6b82cbf4ae8c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:45.920702 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b41197-137c-448a-b145-6b82cbf4ae8c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:45.925415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5bc62-81cb-4d1b-acc7-5162f59121cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:45.926058 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5bc62-81cb-4d1b-acc7-5162f59121cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:45.926335 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5bc62-81cb-4d1b-acc7-5162f59121cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:45.926636 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5bc62-81cb-4d1b-acc7-5162f59121cb admin admin] PATH_INFO: `/v3/projects/1f22f3cfd46141c3a556f50a9fa968a0/users/8bdbfe39015b4869ab6f9a8acc4e6fad/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:45.941744 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5bc62-81cb-4d1b-acc7-5162f59121cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f22f3cfd46141c3a556f50a9fa968a0, user_id=8bdbfe39015b4869ab6f9a8acc4e6fad, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:45.944459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5bc62-81cb-4d1b-acc7-5162f59121cb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:46.057513 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1177/2335] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:20:45 2022] PUT /identity/v3/projects/1f22f3cfd46141c3a556f50a9fa968a0/users/8bdbfe39015b4869ab6f9a8acc4e6fad/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:20:46.061604 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6eadae-0844-4fa5-82c7-73f60f6c2d22 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:46.072007 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6eadae-0844-4fa5-82c7-73f60f6c2d22 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:46.129017 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e49e6-4ac5-436d-9f2f-6f0010e03f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:46.129557 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e49e6-4ac5-436d-9f2f-6f0010e03f1f admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:46.129890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e49e6-4ac5-436d-9f2f-6f0010e03f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:46.130196 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e49e6-4ac5-436d-9f2f-6f0010e03f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:46.130845 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e49e6-4ac5-436d-9f2f-6f0010e03f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:46.132437 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e49e6-4ac5-436d-9f2f-6f0010e03f1f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:46.133940 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:46.143496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1159/2336] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:20:46.147875 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5bc62-81cb-4d1b-acc7-5162f59121cb admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:46.154496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:46.158935 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5bc62-81cb-4d1b-acc7-5162f59121cb admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:46.163496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0e4a7-6eea-4668-9056-ebdf544a8089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:46.164142 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0e4a7-6eea-4668-9056-ebdf544a8089 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:46.164414 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0e4a7-6eea-4668-9056-ebdf544a8089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:46.164683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0e4a7-6eea-4668-9056-ebdf544a8089 admin admin] PATH_INFO: `/v3/projects/1f22f3cfd46141c3a556f50a9fa968a0/users/8bdbfe39015b4869ab6f9a8acc4e6fad/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:46.166641 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c0e4a7-6eea-4668-9056-ebdf544a8089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f22f3cfd46141c3a556f50a9fa968a0, user_id=8bdbfe39015b4869ab6f9a8acc4e6fad, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:46.169342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c0e4a7-6eea-4668-9056-ebdf544a8089 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:46.179981 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1178/2337] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:20:46 2022] PUT /identity/v3/projects/1f22f3cfd46141c3a556f50a9fa968a0/users/8bdbfe39015b4869ab6f9a8acc4e6fad/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:20:46.191158 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79d71b-3408-4ee5-9c59-29b9ec07db5b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:46.191442 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79d71b-3408-4ee5-9c59-29b9ec07db5b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:46.191737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79d71b-3408-4ee5-9c59-29b9ec07db5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:46.220614 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-9b79d71b-3408-4ee5-9c59-29b9ec07db5b None None] MFA Rules not processed for user `8bdbfe39015b4869ab6f9a8acc4e6fad`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:20:46.221827 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9b79d71b-3408-4ee5-9c59-29b9ec07db5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:46.288610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1160/2338] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:20:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:20:46.300717 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0e4a7-6eea-4668-9056-ebdf544a8089 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:46.302422 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-01c0e4a7-6eea-4668-9056-ebdf544a8089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:46.359519 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0e4a7-6eea-4668-9056-ebdf544a8089 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:46.363144 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922ede1-3a0c-44eb-9467-7a41a1971532 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:46.363749 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922ede1-3a0c-44eb-9467-7a41a1971532 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:46.364025 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922ede1-3a0c-44eb-9467-7a41a1971532 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:46.364291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922ede1-3a0c-44eb-9467-7a41a1971532 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:46.373653 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7922ede1-3a0c-44eb-9467-7a41a1971532 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:46.375737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7922ede1-3a0c-44eb-9467-7a41a1971532 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:46.385597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1179/2339] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:20:46 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:20:46.649388 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79d71b-3408-4ee5-9c59-29b9ec07db5b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:46.659166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79d71b-3408-4ee5-9c59-29b9ec07db5b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:46.662763 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9550264-91eb-45ce-b3d2-1b7e08cc1979 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:46.663347 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9550264-91eb-45ce-b3d2-1b7e08cc1979 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:46.663622 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9550264-91eb-45ce-b3d2-1b7e08cc1979 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:46.663894 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9550264-91eb-45ce-b3d2-1b7e08cc1979 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:46.722122 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9550264-91eb-45ce-b3d2-1b7e08cc1979 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:46.723593 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9550264-91eb-45ce-b3d2-1b7e08cc1979 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:46.732433 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1161/2340] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:20:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:20:50.413786 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922ede1-3a0c-44eb-9467-7a41a1971532 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:50.477388 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922ede1-3a0c-44eb-9467-7a41a1971532 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:50.481981 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee577290-17eb-49af-8f4f-ea3ebe3b7c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:50.482631 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee577290-17eb-49af-8f4f-ea3ebe3b7c9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:50.482908 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee577290-17eb-49af-8f4f-ea3ebe3b7c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:50.483181 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee577290-17eb-49af-8f4f-ea3ebe3b7c9f admin admin] PATH_INFO: `/v3/users/0d6c690821b74bcc99a6fb32cc59aa11` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:50.485410 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee577290-17eb-49af-8f4f-ea3ebe3b7c9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d6c690821b74bcc99a6fb32cc59aa11)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:50.488656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee577290-17eb-49af-8f4f-ea3ebe3b7c9f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:50.708796 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ee577290-17eb-49af-8f4f-ea3ebe3b7c9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d6c690821b74bcc99a6fb32cc59aa11'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:50.816852 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ee577290-17eb-49af-8f4f-ea3ebe3b7c9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d6c690821b74bcc99a6fb32cc59aa11'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:50.824465 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ee577290-17eb-49af-8f4f-ea3ebe3b7c9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d6c690821b74bcc99a6fb32cc59aa11'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:50.838114 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ee577290-17eb-49af-8f4f-ea3ebe3b7c9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d6c690821b74bcc99a6fb32cc59aa11'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:50.838842 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ee577290-17eb-49af-8f4f-ea3ebe3b7c9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d6c690821b74bcc99a6fb32cc59aa11'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:50.845567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1180/2341] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:20:50 2022] DELETE /identity/v3/users/0d6c690821b74bcc99a6fb32cc59aa11 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:20:50.850260 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b6b8b-f821-4c0d-8262-aa4dd61a2f78 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:50.856625 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b6b8b-f821-4c0d-8262-aa4dd61a2f78 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:50.856625 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b6b8b-f821-4c0d-8262-aa4dd61a2f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:50.871756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f85b6b8b-f821-4c0d-8262-aa4dd61a2f78 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:20:50.873023 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f85b6b8b-f821-4c0d-8262-aa4dd61a2f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:50.936151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1162/2342] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:20:50.946538 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee577290-17eb-49af-8f4f-ea3ebe3b7c9f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:50.947996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ee577290-17eb-49af-8f4f-ea3ebe3b7c9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:50.962272 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:51.006010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee577290-17eb-49af-8f4f-ea3ebe3b7c9f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:51.010127 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d2d28-3cd4-4f70-91f6-d88df8d2caa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:51.010765 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d2d28-3cd4-4f70-91f6-d88df8d2caa4 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:51.011040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d2d28-3cd4-4f70-91f6-d88df8d2caa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:51.011306 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d2d28-3cd4-4f70-91f6-d88df8d2caa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:51.020126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d2d28-3cd4-4f70-91f6-d88df8d2caa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:51.021574 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d2d28-3cd4-4f70-91f6-d88df8d2caa4 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:51.032862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1181/2343] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:20:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:20:51.470636 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b6b8b-f821-4c0d-8262-aa4dd61a2f78 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:51.472054 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f85b6b8b-f821-4c0d-8262-aa4dd61a2f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:51.485126 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b6b8b-f821-4c0d-8262-aa4dd61a2f78 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:51.490206 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de03031d-377e-4e99-8da6-59d0e6ca095d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:51.490708 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03031d-377e-4e99-8da6-59d0e6ca095d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:51.491051 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03031d-377e-4e99-8da6-59d0e6ca095d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:51.491394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03031d-377e-4e99-8da6-59d0e6ca095d admin admin] PATH_INFO: `/v3/projects/b3ce06fb3a0e4696b4a0ffed8859deca` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:51.492619 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de03031d-377e-4e99-8da6-59d0e6ca095d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3ce06fb3a0e4696b4a0ffed8859deca)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:51.494939 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de03031d-377e-4e99-8da6-59d0e6ca095d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:51.513743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:51.600842 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-de03031d-377e-4e99-8da6-59d0e6ca095d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3ce06fb3a0e4696b4a0ffed8859deca', 'request_id': 'req-de03031d-377e-4e99-8da6-59d0e6ca095d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:51.662733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-de03031d-377e-4e99-8da6-59d0e6ca095d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3ce06fb3a0e4696b4a0ffed8859deca', 'request_id': 'req-de03031d-377e-4e99-8da6-59d0e6ca095d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:51.669632 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-de03031d-377e-4e99-8da6-59d0e6ca095d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3ce06fb3a0e4696b4a0ffed8859deca', 'request_id': 'req-de03031d-377e-4e99-8da6-59d0e6ca095d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:51.680319 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-de03031d-377e-4e99-8da6-59d0e6ca095d admin admin] The token cache is being invalidate because project b3ce06fb3a0e4696b4a0ffed8859deca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:20:51.680601 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-de03031d-377e-4e99-8da6-59d0e6ca095d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:51.681228 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-de03031d-377e-4e99-8da6-59d0e6ca095d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:51.682736 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1163/2344] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:20:51 2022] DELETE /identity/v3/projects/b3ce06fb3a0e4696b4a0ffed8859deca => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:20:51.687026 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d2d28-3cd4-4f70-91f6-d88df8d2caa4 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:51.697169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d2d28-3cd4-4f70-91f6-d88df8d2caa4 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:51.756008 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d3367-5067-4f0a-b477-d4349702ae48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:51.756640 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d3367-5067-4f0a-b477-d4349702ae48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:51.756919 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d3367-5067-4f0a-b477-d4349702ae48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:51.757188 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d3367-5067-4f0a-b477-d4349702ae48 admin admin] PATH_INFO: `/v3/users/8bdbfe39015b4869ab6f9a8acc4e6fad` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:51.758599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8d3367-5067-4f0a-b477-d4349702ae48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bdbfe39015b4869ab6f9a8acc4e6fad)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:51.760523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8d3367-5067-4f0a-b477-d4349702ae48 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:51.974442 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4f8d3367-5067-4f0a-b477-d4349702ae48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bdbfe39015b4869ab6f9a8acc4e6fad'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:52.063094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4f8d3367-5067-4f0a-b477-d4349702ae48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bdbfe39015b4869ab6f9a8acc4e6fad'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:52.070521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4f8d3367-5067-4f0a-b477-d4349702ae48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bdbfe39015b4869ab6f9a8acc4e6fad'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:52.083470 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4f8d3367-5067-4f0a-b477-d4349702ae48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bdbfe39015b4869ab6f9a8acc4e6fad'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:52.084319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4f8d3367-5067-4f0a-b477-d4349702ae48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bdbfe39015b4869ab6f9a8acc4e6fad'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:52.090299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1182/2345] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:20:51 2022] DELETE /identity/v3/users/8bdbfe39015b4869ab6f9a8acc4e6fad => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:20:52.578570 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de03031d-377e-4e99-8da6-59d0e6ca095d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:52.580391 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-de03031d-377e-4e99-8da6-59d0e6ca095d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:52.594999 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de03031d-377e-4e99-8da6-59d0e6ca095d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:52.665078 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75046f87-1e20-4437-b71e-602a8eea0002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:52.665707 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75046f87-1e20-4437-b71e-602a8eea0002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:52.665982 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75046f87-1e20-4437-b71e-602a8eea0002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:52.666252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75046f87-1e20-4437-b71e-602a8eea0002 admin admin] PATH_INFO: `/v3/projects/1f22f3cfd46141c3a556f50a9fa968a0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:52.667459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75046f87-1e20-4437-b71e-602a8eea0002 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f22f3cfd46141c3a556f50a9fa968a0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:52.669364 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75046f87-1e20-4437-b71e-602a8eea0002 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:52.866884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-75046f87-1e20-4437-b71e-602a8eea0002 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f22f3cfd46141c3a556f50a9fa968a0', 'request_id': 'req-75046f87-1e20-4437-b71e-602a8eea0002'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:52.976156 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-75046f87-1e20-4437-b71e-602a8eea0002 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f22f3cfd46141c3a556f50a9fa968a0', 'request_id': 'req-75046f87-1e20-4437-b71e-602a8eea0002'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:52.984673 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-75046f87-1e20-4437-b71e-602a8eea0002 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f22f3cfd46141c3a556f50a9fa968a0', 'request_id': 'req-75046f87-1e20-4437-b71e-602a8eea0002'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:52.995849 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-75046f87-1e20-4437-b71e-602a8eea0002 admin admin] The token cache is being invalidate because project 1f22f3cfd46141c3a556f50a9fa968a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:20:52.996137 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-75046f87-1e20-4437-b71e-602a8eea0002 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:52.996790 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-75046f87-1e20-4437-b71e-602a8eea0002 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:52.998398 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1164/2346] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:20:52 2022] DELETE /identity/v3/projects/1f22f3cfd46141c3a556f50a9fa968a0 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:20:53.004924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfc471-504b-4072-8aec-97cc5134aa3c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:53.005563 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfc471-504b-4072-8aec-97cc5134aa3c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:53.005563 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfc471-504b-4072-8aec-97cc5134aa3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:53.025015 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1bdfc471-504b-4072-8aec-97cc5134aa3c None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:20:53.026196 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1bdfc471-504b-4072-8aec-97cc5134aa3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:53.056322 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:53.084940 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1183/2347] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:20:53.092806 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c6a09-ba9a-4ac7-ba62-fb6f02283c86 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:53.093173 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c6a09-ba9a-4ac7-ba62-fb6f02283c86 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:53.093473 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c6a09-ba9a-4ac7-ba62-fb6f02283c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:53.114050 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-291c6a09-ba9a-4ac7-ba62-fb6f02283c86 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:20:53.115296 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-291c6a09-ba9a-4ac7-ba62-fb6f02283c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:53.124240 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1165/2348] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:20:53.129262 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfc471-504b-4072-8aec-97cc5134aa3c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:53.139795 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfc471-504b-4072-8aec-97cc5134aa3c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:53.144657 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc9ff6-38bc-4353-aa09-313109048b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:53.145246 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc9ff6-38bc-4353-aa09-313109048b28 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:53.148448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc9ff6-38bc-4353-aa09-313109048b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:53.148448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc9ff6-38bc-4353-aa09-313109048b28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:53.148448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadc9ff6-38bc-4353-aa09-313109048b28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:20:53.148448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadc9ff6-38bc-4353-aa09-313109048b28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:53.148778 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadc9ff6-38bc-4353-aa09-313109048b28 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:53.160137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1184/2349] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:20:53.164729 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c6a09-ba9a-4ac7-ba62-fb6f02283c86 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:53.175381 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c6a09-ba9a-4ac7-ba62-fb6f02283c86 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:53.180635 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327aa0fa-d725-49f2-9e5d-0978ca4c06f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:53.181230 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327aa0fa-d725-49f2-9e5d-0978ca4c06f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:53.181503 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327aa0fa-d725-49f2-9e5d-0978ca4c06f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:53.181772 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327aa0fa-d725-49f2-9e5d-0978ca4c06f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:53.182491 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327aa0fa-d725-49f2-9e5d-0978ca4c06f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:53.184413 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327aa0fa-d725-49f2-9e5d-0978ca4c06f3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:53.282852 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1166/2350] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:20:53 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:20:53.286830 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc9ff6-38bc-4353-aa09-313109048b28 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:53.297420 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc9ff6-38bc-4353-aa09-313109048b28 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:53.352389 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6af285-2401-48b2-967e-f794e657bb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:53.352915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6af285-2401-48b2-967e-f794e657bb4e admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:53.353189 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6af285-2401-48b2-967e-f794e657bb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:53.353459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6af285-2401-48b2-967e-f794e657bb4e admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:53.354113 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6af285-2401-48b2-967e-f794e657bb4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:53.356061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6af285-2401-48b2-967e-f794e657bb4e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:53.474165 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1185/2351] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:20:53 2022] POST /identity/v3/users => generated 294 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:20:53.478567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327aa0fa-d725-49f2-9e5d-0978ca4c06f3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:53.486100 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:53.489939 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327aa0fa-d725-49f2-9e5d-0978ca4c06f3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:53.494914 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa5d9b-a43c-4965-a6a5-08729544f471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:53.497827 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa5d9b-a43c-4965-a6a5-08729544f471 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:53.497827 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa5d9b-a43c-4965-a6a5-08729544f471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:53.497827 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa5d9b-a43c-4965-a6a5-08729544f471 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:53.497827 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa5d9b-a43c-4965-a6a5-08729544f471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:53.498339 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa5d9b-a43c-4965-a6a5-08729544f471 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:53.510104 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1167/2352] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:20:53.519982 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6af285-2401-48b2-967e-f794e657bb4e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:53.530070 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6af285-2401-48b2-967e-f794e657bb4e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:53.535858 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc2adbc-ed60-42c5-87c4-59d0dcafc6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:53.536581 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2adbc-ed60-42c5-87c4-59d0dcafc6aa admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:53.536889 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2adbc-ed60-42c5-87c4-59d0dcafc6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:53.537149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2adbc-ed60-42c5-87c4-59d0dcafc6aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:53.537790 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc2adbc-ed60-42c5-87c4-59d0dcafc6aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:53.539593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc2adbc-ed60-42c5-87c4-59d0dcafc6aa admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:53.552377 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1186/2353] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:20:53.556602 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa5d9b-a43c-4965-a6a5-08729544f471 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:53.567248 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa5d9b-a43c-4965-a6a5-08729544f471 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:53.572599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb253cb-11a6-4b65-9ab2-6b1d024e5145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:53.573492 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb253cb-11a6-4b65-9ab2-6b1d024e5145 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:53.573881 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb253cb-11a6-4b65-9ab2-6b1d024e5145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:53.574257 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb253cb-11a6-4b65-9ab2-6b1d024e5145 admin admin] PATH_INFO: `/v3/projects/69d868122ba1489891b68222be8a947d/users/df81f4f8949c4640b0d581fe99b4a92e/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:53.590053 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb253cb-11a6-4b65-9ab2-6b1d024e5145 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69d868122ba1489891b68222be8a947d, user_id=df81f4f8949c4640b0d581fe99b4a92e, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:53.592741 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb253cb-11a6-4b65-9ab2-6b1d024e5145 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:53.697575 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1168/2354] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:20:53 2022] PUT /identity/v3/projects/69d868122ba1489891b68222be8a947d/users/df81f4f8949c4640b0d581fe99b4a92e/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:20:53.716534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c18a2c-c6b5-4221-8e69-65e5612a0503 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:53.716832 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c18a2c-c6b5-4221-8e69-65e5612a0503 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:53.717103 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c18a2c-c6b5-4221-8e69-65e5612a0503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:53.745475 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f9c18a2c-c6b5-4221-8e69-65e5612a0503 None None] MFA Rules not processed for user `df81f4f8949c4640b0d581fe99b4a92e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:20:53.746714 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f9c18a2c-c6b5-4221-8e69-65e5612a0503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:53.815273 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1187/2355] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:20:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:20:53.820257 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb253cb-11a6-4b65-9ab2-6b1d024e5145 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:53.882735 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb253cb-11a6-4b65-9ab2-6b1d024e5145 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:53.887599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7f6eb-cf65-4ae4-a02b-1a30ea0735d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:53.888203 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7f6eb-cf65-4ae4-a02b-1a30ea0735d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:53.888477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7f6eb-cf65-4ae4-a02b-1a30ea0735d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:53.888843 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7f6eb-cf65-4ae4-a02b-1a30ea0735d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:53.889378 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff7f6eb-cf65-4ae4-a02b-1a30ea0735d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:53.891930 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff7f6eb-cf65-4ae4-a02b-1a30ea0735d8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:54.070408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1169/2356] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:20:53 2022] POST /identity/v3/projects => generated 387 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:20:54.074585 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c18a2c-c6b5-4221-8e69-65e5612a0503 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:54.086172 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c18a2c-c6b5-4221-8e69-65e5612a0503 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:54.115132 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:54.144678 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454892a6-6d9c-4811-828e-0320dbd3528a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:54.145200 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454892a6-6d9c-4811-828e-0320dbd3528a admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:54.145476 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454892a6-6d9c-4811-828e-0320dbd3528a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:54.145745 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454892a6-6d9c-4811-828e-0320dbd3528a admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:54.146381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454892a6-6d9c-4811-828e-0320dbd3528a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:54.148445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454892a6-6d9c-4811-828e-0320dbd3528a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:54.288872 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1188/2357] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:20:54 2022] POST /identity/v3/users => generated 300 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:20:54.290008 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7f6eb-cf65-4ae4-a02b-1a30ea0735d8 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:54.300864 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7f6eb-cf65-4ae4-a02b-1a30ea0735d8 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:54.305893 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511281dd-fb4e-462a-907a-160d401344bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:54.306545 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511281dd-fb4e-462a-907a-160d401344bc admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:54.306832 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511281dd-fb4e-462a-907a-160d401344bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:54.307107 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511281dd-fb4e-462a-907a-160d401344bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:54.307760 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511281dd-fb4e-462a-907a-160d401344bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:54.309471 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511281dd-fb4e-462a-907a-160d401344bc admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:54.321304 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1170/2358] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:20:54.332666 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454892a6-6d9c-4811-828e-0320dbd3528a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:54.342937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454892a6-6d9c-4811-828e-0320dbd3528a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:54.348005 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a253b248-ad64-4c4d-9bb0-5f78f7e8131a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:54.348642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253b248-ad64-4c4d-9bb0-5f78f7e8131a admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:54.348917 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253b248-ad64-4c4d-9bb0-5f78f7e8131a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:54.349184 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253b248-ad64-4c4d-9bb0-5f78f7e8131a admin admin] PATH_INFO: `/v3/projects/9d64ee5d4c4a4a7eb88e776ec4781561/users/15c20156a62a4a8eb74b178059c37fd0/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:54.364740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a253b248-ad64-4c4d-9bb0-5f78f7e8131a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d64ee5d4c4a4a7eb88e776ec4781561, user_id=15c20156a62a4a8eb74b178059c37fd0, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:54.367609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a253b248-ad64-4c4d-9bb0-5f78f7e8131a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:54.441157 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1189/2359] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:20:54 2022] PUT /identity/v3/projects/9d64ee5d4c4a4a7eb88e776ec4781561/users/15c20156a62a4a8eb74b178059c37fd0/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:20:54.445197 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511281dd-fb4e-462a-907a-160d401344bc admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:54.456432 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511281dd-fb4e-462a-907a-160d401344bc admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:54.515541 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eac6d3-be1f-4c0e-9806-c9ef33f9fb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:54.516134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eac6d3-be1f-4c0e-9806-c9ef33f9fb57 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:54.516408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eac6d3-be1f-4c0e-9806-c9ef33f9fb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:54.516669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eac6d3-be1f-4c0e-9806-c9ef33f9fb57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:54.517305 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eac6d3-be1f-4c0e-9806-c9ef33f9fb57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:54.518960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eac6d3-be1f-4c0e-9806-c9ef33f9fb57 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:54.529038 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1171/2360] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:20:54.533513 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a253b248-ad64-4c4d-9bb0-5f78f7e8131a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:54.543783 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a253b248-ad64-4c4d-9bb0-5f78f7e8131a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:54.549172 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a523d-8900-4baf-b264-53a1c81ccf62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:54.549824 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a523d-8900-4baf-b264-53a1c81ccf62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:54.550083 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a523d-8900-4baf-b264-53a1c81ccf62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:54.550427 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a523d-8900-4baf-b264-53a1c81ccf62 admin admin] PATH_INFO: `/v3/projects/9d64ee5d4c4a4a7eb88e776ec4781561/users/15c20156a62a4a8eb74b178059c37fd0/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:54.552287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a523d-8900-4baf-b264-53a1c81ccf62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d64ee5d4c4a4a7eb88e776ec4781561, user_id=15c20156a62a4a8eb74b178059c37fd0, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:54.554976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a523d-8900-4baf-b264-53a1c81ccf62 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:54.565844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1190/2361] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:20:54 2022] PUT /identity/v3/projects/9d64ee5d4c4a4a7eb88e776ec4781561/users/15c20156a62a4a8eb74b178059c37fd0/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:20:54.575922 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377cf755-1fa3-487f-9a36-bfcd52012e25 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:54.576232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377cf755-1fa3-487f-9a36-bfcd52012e25 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:54.576508 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377cf755-1fa3-487f-9a36-bfcd52012e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:54.607019 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-377cf755-1fa3-487f-9a36-bfcd52012e25 None None] MFA Rules not processed for user `15c20156a62a4a8eb74b178059c37fd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:20:54.608237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-377cf755-1fa3-487f-9a36-bfcd52012e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:54.640284 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:54.678454 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1172/2362] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:20:54 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:20:54.690092 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a523d-8900-4baf-b264-53a1c81ccf62 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:54.691955 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-443a523d-8900-4baf-b264-53a1c81ccf62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:54.756046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a523d-8900-4baf-b264-53a1c81ccf62 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:54.759987 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338e130-4296-4210-9d4b-684f1ccdf57c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:54.760591 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338e130-4296-4210-9d4b-684f1ccdf57c service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:54.760871 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338e130-4296-4210-9d4b-684f1ccdf57c service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:54.761140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338e130-4296-4210-9d4b-684f1ccdf57c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:54.770397 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338e130-4296-4210-9d4b-684f1ccdf57c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:54.771902 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338e130-4296-4210-9d4b-684f1ccdf57c service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:54.782319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1191/2363] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:20:54 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:20:54.821068 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377cf755-1fa3-487f-9a36-bfcd52012e25 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:54.889134 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377cf755-1fa3-487f-9a36-bfcd52012e25 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:54.894372 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa1b6b-cab2-4b92-b51f-bc689dbd90f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:54.894993 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa1b6b-cab2-4b92-b51f-bc689dbd90f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:54.895273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa1b6b-cab2-4b92-b51f-bc689dbd90f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:54.895542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa1b6b-cab2-4b92-b51f-bc689dbd90f1 admin admin] PATH_INFO: `/v3/users/df81f4f8949c4640b0d581fe99b4a92e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:54.896670 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa1b6b-cab2-4b92-b51f-bc689dbd90f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df81f4f8949c4640b0d581fe99b4a92e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:54.898622 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa1b6b-cab2-4b92-b51f-bc689dbd90f1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:55.140569 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-31aa1b6b-cab2-4b92-b51f-bc689dbd90f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df81f4f8949c4640b0d581fe99b4a92e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:55.246135 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-31aa1b6b-cab2-4b92-b51f-bc689dbd90f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df81f4f8949c4640b0d581fe99b4a92e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:55.260824 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-31aa1b6b-cab2-4b92-b51f-bc689dbd90f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df81f4f8949c4640b0d581fe99b4a92e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:55.261557 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-31aa1b6b-cab2-4b92-b51f-bc689dbd90f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df81f4f8949c4640b0d581fe99b4a92e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:55.262311 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-31aa1b6b-cab2-4b92-b51f-bc689dbd90f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df81f4f8949c4640b0d581fe99b4a92e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:55.277871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1173/2364] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:20:54 2022] DELETE /identity/v3/users/df81f4f8949c4640b0d581fe99b4a92e => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:20:55.283354 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841d87a-31f3-411d-b5cd-b1f3bf2bbf43 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:55.283811 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841d87a-31f3-411d-b5cd-b1f3bf2bbf43 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:55.283973 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841d87a-31f3-411d-b5cd-b1f3bf2bbf43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:55.302925 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d841d87a-31f3-411d-b5cd-b1f3bf2bbf43 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:20:55.304107 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d841d87a-31f3-411d-b5cd-b1f3bf2bbf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:55.363670 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1192/2365] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:20:55.373412 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa1b6b-cab2-4b92-b51f-bc689dbd90f1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:55.375630 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-31aa1b6b-cab2-4b92-b51f-bc689dbd90f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:55.423290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:55.433909 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa1b6b-cab2-4b92-b51f-bc689dbd90f1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:55.438140 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84268e1d-64c8-402e-9664-b23c5b09aa34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:55.438768 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84268e1d-64c8-402e-9664-b23c5b09aa34 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:55.439044 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84268e1d-64c8-402e-9664-b23c5b09aa34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:55.439312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84268e1d-64c8-402e-9664-b23c5b09aa34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:55.448408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84268e1d-64c8-402e-9664-b23c5b09aa34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:55.449839 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84268e1d-64c8-402e-9664-b23c5b09aa34 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:55.460992 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1174/2366] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:20:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:20:55.917368 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841d87a-31f3-411d-b5cd-b1f3bf2bbf43 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:55.919176 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d841d87a-31f3-411d-b5cd-b1f3bf2bbf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:55.940703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841d87a-31f3-411d-b5cd-b1f3bf2bbf43 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:55.947034 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86341e89-0030-4d18-8640-5e0a1735fcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:55.947719 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86341e89-0030-4d18-8640-5e0a1735fcd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:55.948019 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86341e89-0030-4d18-8640-5e0a1735fcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:55.948317 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86341e89-0030-4d18-8640-5e0a1735fcd2 admin admin] PATH_INFO: `/v3/projects/69d868122ba1489891b68222be8a947d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:55.949459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86341e89-0030-4d18-8640-5e0a1735fcd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69d868122ba1489891b68222be8a947d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:55.951501 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86341e89-0030-4d18-8640-5e0a1735fcd2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:55.977069 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:56.103592 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-86341e89-0030-4d18-8640-5e0a1735fcd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69d868122ba1489891b68222be8a947d', 'request_id': 'req-86341e89-0030-4d18-8640-5e0a1735fcd2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:56.110982 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-86341e89-0030-4d18-8640-5e0a1735fcd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69d868122ba1489891b68222be8a947d', 'request_id': 'req-86341e89-0030-4d18-8640-5e0a1735fcd2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:56.176789 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-86341e89-0030-4d18-8640-5e0a1735fcd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69d868122ba1489891b68222be8a947d', 'request_id': 'req-86341e89-0030-4d18-8640-5e0a1735fcd2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:56.188435 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-86341e89-0030-4d18-8640-5e0a1735fcd2 admin admin] The token cache is being invalidate because project 69d868122ba1489891b68222be8a947d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:20:56.188711 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-86341e89-0030-4d18-8640-5e0a1735fcd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:56.189330 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-86341e89-0030-4d18-8640-5e0a1735fcd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:56.190905 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1193/2367] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:20:55 2022] DELETE /identity/v3/projects/69d868122ba1489891b68222be8a947d => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:20:56.195003 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84268e1d-64c8-402e-9664-b23c5b09aa34 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:56.205103 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84268e1d-64c8-402e-9664-b23c5b09aa34 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:56.261871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e9f26-f744-48af-b07b-73451a2ed74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:56.262437 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e9f26-f744-48af-b07b-73451a2ed74d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:56.262713 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e9f26-f744-48af-b07b-73451a2ed74d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:56.262984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e9f26-f744-48af-b07b-73451a2ed74d admin admin] PATH_INFO: `/v3/users/15c20156a62a4a8eb74b178059c37fd0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:56.264103 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e9f26-f744-48af-b07b-73451a2ed74d admin admin] RBAC: Authorizing `identity:delete_user(user_id=15c20156a62a4a8eb74b178059c37fd0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:56.266181 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e9f26-f744-48af-b07b-73451a2ed74d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:56.473222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-781e9f26-f744-48af-b07b-73451a2ed74d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15c20156a62a4a8eb74b178059c37fd0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:56.554264 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-781e9f26-f744-48af-b07b-73451a2ed74d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15c20156a62a4a8eb74b178059c37fd0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:56.568556 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-781e9f26-f744-48af-b07b-73451a2ed74d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15c20156a62a4a8eb74b178059c37fd0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:56.569277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-781e9f26-f744-48af-b07b-73451a2ed74d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15c20156a62a4a8eb74b178059c37fd0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:56.569904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-781e9f26-f744-48af-b07b-73451a2ed74d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15c20156a62a4a8eb74b178059c37fd0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:56.582674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1175/2368] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:20:56 2022] DELETE /identity/v3/users/15c20156a62a4a8eb74b178059c37fd0 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:20:57.007408 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86341e89-0030-4d18-8640-5e0a1735fcd2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:57.009148 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-86341e89-0030-4d18-8640-5e0a1735fcd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:57.023313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86341e89-0030-4d18-8640-5e0a1735fcd2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:57.082089 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0a62f-b34c-4cd1-b4d2-8e5d71e5d250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:57.082748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0a62f-b34c-4cd1-b4d2-8e5d71e5d250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:57.083216 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0a62f-b34c-4cd1-b4d2-8e5d71e5d250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:57.083490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0a62f-b34c-4cd1-b4d2-8e5d71e5d250 admin admin] PATH_INFO: `/v3/projects/9d64ee5d4c4a4a7eb88e776ec4781561` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:57.084601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0a62f-b34c-4cd1-b4d2-8e5d71e5d250 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d64ee5d4c4a4a7eb88e776ec4781561)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:57.086522 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0a62f-b34c-4cd1-b4d2-8e5d71e5d250 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:57.246640 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1db0a62f-b34c-4cd1-b4d2-8e5d71e5d250 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d64ee5d4c4a4a7eb88e776ec4781561', 'request_id': 'req-1db0a62f-b34c-4cd1-b4d2-8e5d71e5d250'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:57.253463 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1db0a62f-b34c-4cd1-b4d2-8e5d71e5d250 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d64ee5d4c4a4a7eb88e776ec4781561', 'request_id': 'req-1db0a62f-b34c-4cd1-b4d2-8e5d71e5d250'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:57.343655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1db0a62f-b34c-4cd1-b4d2-8e5d71e5d250 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d64ee5d4c4a4a7eb88e776ec4781561', 'request_id': 'req-1db0a62f-b34c-4cd1-b4d2-8e5d71e5d250'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:57.355060 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1db0a62f-b34c-4cd1-b4d2-8e5d71e5d250 admin admin] The token cache is being invalidate because project 9d64ee5d4c4a4a7eb88e776ec4781561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:20:57.355350 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1db0a62f-b34c-4cd1-b4d2-8e5d71e5d250 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:57.355990 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1db0a62f-b34c-4cd1-b4d2-8e5d71e5d250 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:57.357686 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1194/2369] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:20:57 2022] DELETE /identity/v3/projects/9d64ee5d4c4a4a7eb88e776ec4781561 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:20:57.363433 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3d77c-82e4-4805-a592-a2f2d2f3ef55 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:57.364136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3d77c-82e4-4805-a592-a2f2d2f3ef55 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:57.364295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3d77c-82e4-4805-a592-a2f2d2f3ef55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:57.385498 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-32c3d77c-82e4-4805-a592-a2f2d2f3ef55 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:20:57.386809 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-32c3d77c-82e4-4805-a592-a2f2d2f3ef55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:57.453545 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1176/2370] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:20:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:20:57.462169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fb922-0b32-497a-98f9-047e421f05e9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:57.462483 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fb922-0b32-497a-98f9-047e421f05e9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:57.462760 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fb922-0b32-497a-98f9-047e421f05e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:57.483673 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5a5fb922-0b32-497a-98f9-047e421f05e9 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:20:57.485401 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5a5fb922-0b32-497a-98f9-047e421f05e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:57.493836 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1195/2371] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:20:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:20:57.499284 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3d77c-82e4-4805-a592-a2f2d2f3ef55 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:57.510597 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3d77c-82e4-4805-a592-a2f2d2f3ef55 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:57.515047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec1068-c1e4-4606-b885-34149b615a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:57.515635 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec1068-c1e4-4606-b885-34149b615a53 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:57.515908 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec1068-c1e4-4606-b885-34149b615a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:57.516174 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec1068-c1e4-4606-b885-34149b615a53 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:57.516959 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec1068-c1e4-4606-b885-34149b615a53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:20:57.517295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec1068-c1e4-4606-b885-34149b615a53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:57.518940 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec1068-c1e4-4606-b885-34149b615a53 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:57.523383 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:57.531380 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1177/2372] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:20:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:20:57.536102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5fb922-0b32-497a-98f9-047e421f05e9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:57.545514 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5fb922-0b32-497a-98f9-047e421f05e9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:57.551061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21736d9a-cc5d-46d7-a6e6-2bb6fdd8cb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:57.551765 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21736d9a-cc5d-46d7-a6e6-2bb6fdd8cb72 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:57.551955 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21736d9a-cc5d-46d7-a6e6-2bb6fdd8cb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:57.552327 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21736d9a-cc5d-46d7-a6e6-2bb6fdd8cb72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:57.552888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21736d9a-cc5d-46d7-a6e6-2bb6fdd8cb72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:57.554905 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21736d9a-cc5d-46d7-a6e6-2bb6fdd8cb72 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:57.652803 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1196/2373] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:20:57 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:20:57.657952 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec1068-c1e4-4606-b885-34149b615a53 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:57.667758 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec1068-c1e4-4606-b885-34149b615a53 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:57.722491 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3677d-fa4a-42c9-afe1-f3ad963ef097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:57.723082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3677d-fa4a-42c9-afe1-f3ad963ef097 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:57.723357 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3677d-fa4a-42c9-afe1-f3ad963ef097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:57.723620 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3677d-fa4a-42c9-afe1-f3ad963ef097 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:57.724235 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a3677d-fa4a-42c9-afe1-f3ad963ef097 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:57.726589 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a3677d-fa4a-42c9-afe1-f3ad963ef097 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:57.817084 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1178/2374] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:20:57 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:20:57.821112 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21736d9a-cc5d-46d7-a6e6-2bb6fdd8cb72 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:57.831796 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21736d9a-cc5d-46d7-a6e6-2bb6fdd8cb72 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:57.836583 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197342f-6f77-4924-86b5-4d0f3acbf86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:57.837173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197342f-6f77-4924-86b5-4d0f3acbf86d admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:57.837453 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197342f-6f77-4924-86b5-4d0f3acbf86d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:57.837718 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197342f-6f77-4924-86b5-4d0f3acbf86d admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:57.838406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197342f-6f77-4924-86b5-4d0f3acbf86d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:57.839999 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197342f-6f77-4924-86b5-4d0f3acbf86d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:57.845843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:57.850981 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1197/2375] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:20:57.855003 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3677d-fa4a-42c9-afe1-f3ad963ef097 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:57.865918 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3677d-fa4a-42c9-afe1-f3ad963ef097 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:57.870517 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05ec70-e360-4299-bfd2-b142d515d00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:57.871130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05ec70-e360-4299-bfd2-b142d515d00d admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:57.871405 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05ec70-e360-4299-bfd2-b142d515d00d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:57.871674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05ec70-e360-4299-bfd2-b142d515d00d admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:57.872320 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f05ec70-e360-4299-bfd2-b142d515d00d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:57.873905 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f05ec70-e360-4299-bfd2-b142d515d00d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:57.884692 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1179/2376] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:20:57.893120 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197342f-6f77-4924-86b5-4d0f3acbf86d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:57.903165 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197342f-6f77-4924-86b5-4d0f3acbf86d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:57.908467 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46df7e1-fa70-470b-b24a-26a770843fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:57.909114 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46df7e1-fa70-470b-b24a-26a770843fe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:57.909390 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46df7e1-fa70-470b-b24a-26a770843fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:57.909662 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46df7e1-fa70-470b-b24a-26a770843fe0 admin admin] PATH_INFO: `/v3/projects/60ab02421cf946399dedae6c5ec0c9d1/users/76036bf82cbe4498a07c9c7f21679fac/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:57.925304 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46df7e1-fa70-470b-b24a-26a770843fe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60ab02421cf946399dedae6c5ec0c9d1, user_id=76036bf82cbe4498a07c9c7f21679fac, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:57.928459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46df7e1-fa70-470b-b24a-26a770843fe0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:58.035015 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1198/2377] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:20:57 2022] PUT /identity/v3/projects/60ab02421cf946399dedae6c5ec0c9d1/users/76036bf82cbe4498a07c9c7f21679fac/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:20:58.046175 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b053639-c5a8-488a-bac3-da4f2cd00dd6 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:58.046489 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b053639-c5a8-488a-bac3-da4f2cd00dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:58.046759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b053639-c5a8-488a-bac3-da4f2cd00dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:58.075047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1b053639-c5a8-488a-bac3-da4f2cd00dd6 None None] MFA Rules not processed for user `76036bf82cbe4498a07c9c7f21679fac`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:20:58.076249 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1b053639-c5a8-488a-bac3-da4f2cd00dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:58.139397 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1180/2378] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:20:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:20:58.151394 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46df7e1-fa70-470b-b24a-26a770843fe0 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:58.153146 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f46df7e1-fa70-470b-b24a-26a770843fe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:58.209639 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46df7e1-fa70-470b-b24a-26a770843fe0 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:58.213299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7758a74-dd93-4540-a7f8-49f6a0b5c5bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:58.213922 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7758a74-dd93-4540-a7f8-49f6a0b5c5bb service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:58.214199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7758a74-dd93-4540-a7f8-49f6a0b5c5bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:58.214502 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7758a74-dd93-4540-a7f8-49f6a0b5c5bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:58.223803 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7758a74-dd93-4540-a7f8-49f6a0b5c5bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:58.225250 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7758a74-dd93-4540-a7f8-49f6a0b5c5bb service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:58.234532 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1199/2379] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:20:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:20:58.253567 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b053639-c5a8-488a-bac3-da4f2cd00dd6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:58.316640 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b053639-c5a8-488a-bac3-da4f2cd00dd6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:58.321281 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd47b15-4542-4419-82a7-27fb5b0db0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:58.321906 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd47b15-4542-4419-82a7-27fb5b0db0b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:58.322182 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd47b15-4542-4419-82a7-27fb5b0db0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:58.322500 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd47b15-4542-4419-82a7-27fb5b0db0b6 admin admin] PATH_INFO: `/v3/users/76036bf82cbe4498a07c9c7f21679fac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:58.323642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd47b15-4542-4419-82a7-27fb5b0db0b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76036bf82cbe4498a07c9c7f21679fac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:58.325624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd47b15-4542-4419-82a7-27fb5b0db0b6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:58.327984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:20:58.552430 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6fd47b15-4542-4419-82a7-27fb5b0db0b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76036bf82cbe4498a07c9c7f21679fac'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:58.634252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6fd47b15-4542-4419-82a7-27fb5b0db0b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76036bf82cbe4498a07c9c7f21679fac'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:58.648415 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6fd47b15-4542-4419-82a7-27fb5b0db0b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76036bf82cbe4498a07c9c7f21679fac'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:58.649132 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6fd47b15-4542-4419-82a7-27fb5b0db0b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76036bf82cbe4498a07c9c7f21679fac'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:58.649822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6fd47b15-4542-4419-82a7-27fb5b0db0b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76036bf82cbe4498a07c9c7f21679fac'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:58.662764 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1181/2380] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:20:58 2022] DELETE /identity/v3/users/76036bf82cbe4498a07c9c7f21679fac => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:20:58.671149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341635d-c945-439f-8319-a284170679fa None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:58.671149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341635d-c945-439f-8319-a284170679fa None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:58.671149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341635d-c945-439f-8319-a284170679fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:58.689167 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4341635d-c945-439f-8319-a284170679fa None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:20:58.690373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4341635d-c945-439f-8319-a284170679fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:58.753366 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1200/2381] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:20:58.763437 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd47b15-4542-4419-82a7-27fb5b0db0b6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:58.768503 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6fd47b15-4542-4419-82a7-27fb5b0db0b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:58.827698 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd47b15-4542-4419-82a7-27fb5b0db0b6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:58.832555 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9699a8b7-297b-49ab-8900-fd8f348ff7e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:58.833166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699a8b7-297b-49ab-8900-fd8f348ff7e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:58.833569 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699a8b7-297b-49ab-8900-fd8f348ff7e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:58.833841 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699a8b7-297b-49ab-8900-fd8f348ff7e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:58.843319 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9699a8b7-297b-49ab-8900-fd8f348ff7e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:58.845487 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9699a8b7-297b-49ab-8900-fd8f348ff7e3 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:58.854839 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1182/2382] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:20:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:20:59.264658 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341635d-c945-439f-8319-a284170679fa None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:20:59.266168 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4341635d-c945-439f-8319-a284170679fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:20:59.278981 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341635d-c945-439f-8319-a284170679fa None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:20:59.283994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79b25a-a020-4256-88db-945758807b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:20:59.284683 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79b25a-a020-4256-88db-945758807b4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:20:59.284958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79b25a-a020-4256-88db-945758807b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:20:59.285348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79b25a-a020-4256-88db-945758807b4e admin admin] PATH_INFO: `/v3/projects/60ab02421cf946399dedae6c5ec0c9d1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:20:59.286338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a79b25a-a020-4256-88db-945758807b4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=60ab02421cf946399dedae6c5ec0c9d1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:20:59.288268 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a79b25a-a020-4256-88db-945758807b4e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:20:59.434778 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5a79b25a-a020-4256-88db-945758807b4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60ab02421cf946399dedae6c5ec0c9d1', 'request_id': 'req-5a79b25a-a020-4256-88db-945758807b4e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:59.441720 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5a79b25a-a020-4256-88db-945758807b4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60ab02421cf946399dedae6c5ec0c9d1', 'request_id': 'req-5a79b25a-a020-4256-88db-945758807b4e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:59.525251 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5a79b25a-a020-4256-88db-945758807b4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60ab02421cf946399dedae6c5ec0c9d1', 'request_id': 'req-5a79b25a-a020-4256-88db-945758807b4e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:59.536409 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5a79b25a-a020-4256-88db-945758807b4e admin admin] The token cache is being invalidate because project 60ab02421cf946399dedae6c5ec0c9d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:20:59.536724 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5a79b25a-a020-4256-88db-945758807b4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:59.537372 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5a79b25a-a020-4256-88db-945758807b4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:20:59.539072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1201/2383] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:20:59 2022] DELETE /identity/v3/projects/60ab02421cf946399dedae6c5ec0c9d1 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:04.024834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0153fff-2c43-4875-898d-a287aa3c7245 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:04.025320 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0153fff-2c43-4875-898d-a287aa3c7245 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:04.025421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0153fff-2c43-4875-898d-a287aa3c7245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:04.045083 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a0153fff-2c43-4875-898d-a287aa3c7245 None None] MFA Rules not processed for user `4f9e1767ec90435f80248a2930081265`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:21:04.047542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a0153fff-2c43-4875-898d-a287aa3c7245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:04.103478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1183/2384] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 10 17:21:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:21:04.249750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79b25a-a020-4256-88db-945758807b4e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:04.251187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5a79b25a-a020-4256-88db-945758807b4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:04.257386 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:04.328767 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79b25a-a020-4256-88db-945758807b4e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:04.340961 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79152184-f81a-47de-8679-ed3d0d2eca29 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4986821ef9f84b99a835e387635aa9a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:04.341572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79152184-f81a-47de-8679-ed3d0d2eca29 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:04.342203 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79152184-f81a-47de-8679-ed3d0d2eca29 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:04.342795 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79152184-f81a-47de-8679-ed3d0d2eca29 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:04.352709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79152184-f81a-47de-8679-ed3d0d2eca29 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:04.354418 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79152184-f81a-47de-8679-ed3d0d2eca29 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:04.363681 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1202/2385] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 10 17:21:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 17:21:14.095089 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=76,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:14.388448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=31 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:21.157875 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0153fff-2c43-4875-898d-a287aa3c7245 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:21.159683 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a0153fff-2c43-4875-898d-a287aa3c7245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:21.218504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0153fff-2c43-4875-898d-a287aa3c7245 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:21.222922 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803bce68-887e-46a8-ade4-70b3598a6169 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:21.223545 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bce68-887e-46a8-ade4-70b3598a6169 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:21.223808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bce68-887e-46a8-ade4-70b3598a6169 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:21.224074 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bce68-887e-46a8-ade4-70b3598a6169 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:21.225813 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-803bce68-887e-46a8-ade4-70b3598a6169 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:21.293677 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803bce68-887e-46a8-ade4-70b3598a6169 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:21.295156 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803bce68-887e-46a8-ade4-70b3598a6169 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:21.304880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1184/2386] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:21:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:21:25.326479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79152184-f81a-47de-8679-ed3d0d2eca29 service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:25.328664 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-79152184-f81a-47de-8679-ed3d0d2eca29 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:25.350563 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79152184-f81a-47de-8679-ed3d0d2eca29 service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:25.355707 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040a9ee-a36d-48ff-a0ed-eb16d4270bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:25.356382 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040a9ee-a36d-48ff-a0ed-eb16d4270bdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:25.356623 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040a9ee-a36d-48ff-a0ed-eb16d4270bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:25.357523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040a9ee-a36d-48ff-a0ed-eb16d4270bdb admin admin] PATH_INFO: `/v3/users/2f74a20789dc4ef697861146252bacba` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:25.358158 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040a9ee-a36d-48ff-a0ed-eb16d4270bdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f74a20789dc4ef697861146252bacba)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:25.360365 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040a9ee-a36d-48ff-a0ed-eb16d4270bdb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:25.626191 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0040a9ee-a36d-48ff-a0ed-eb16d4270bdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f74a20789dc4ef697861146252bacba'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:25.713306 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0040a9ee-a36d-48ff-a0ed-eb16d4270bdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f74a20789dc4ef697861146252bacba'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:25.721406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0040a9ee-a36d-48ff-a0ed-eb16d4270bdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f74a20789dc4ef697861146252bacba'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:25.734552 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0040a9ee-a36d-48ff-a0ed-eb16d4270bdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f74a20789dc4ef697861146252bacba'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:25.735732 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0040a9ee-a36d-48ff-a0ed-eb16d4270bdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f74a20789dc4ef697861146252bacba'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:25.742241 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1203/2387] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:21:25 2022] DELETE /identity/v3/users/2f74a20789dc4ef697861146252bacba => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:26.035804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803bce68-887e-46a8-ade4-70b3598a6169 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:26.050657 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803bce68-887e-46a8-ade4-70b3598a6169 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:26.114242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af6a8c-9b06-4698-8b1f-087ad438e9ef None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:26.114242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af6a8c-9b06-4698-8b1f-087ad438e9ef None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:26.114242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af6a8c-9b06-4698-8b1f-087ad438e9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:26.120725 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f806dcc7-6124-41ad-8fcc-ce7334e0ac39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:26.121701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806dcc7-6124-41ad-8fcc-ce7334e0ac39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:26.122081 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806dcc7-6124-41ad-8fcc-ce7334e0ac39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:26.122496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806dcc7-6124-41ad-8fcc-ce7334e0ac39 admin admin] PATH_INFO: `/v3/projects/7e92e1e1c27241798e336e547580be5e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:26.123491 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-96af6a8c-9b06-4698-8b1f-087ad438e9ef None None] MFA Rules not processed for user `4f9e1767ec90435f80248a2930081265`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:21:26.124525 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f806dcc7-6124-41ad-8fcc-ce7334e0ac39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e92e1e1c27241798e336e547580be5e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:26.124731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-96af6a8c-9b06-4698-8b1f-087ad438e9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:26.126804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f806dcc7-6124-41ad-8fcc-ce7334e0ac39 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:26.186400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1204/2388] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 10 17:21:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:21:26.199115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af6a8c-9b06-4698-8b1f-087ad438e9ef None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:26.200602 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-96af6a8c-9b06-4698-8b1f-087ad438e9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:26.280540 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af6a8c-9b06-4698-8b1f-087ad438e9ef None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:26.291604 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f806dcc7-6124-41ad-8fcc-ce7334e0ac39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e92e1e1c27241798e336e547580be5e', 'request_id': 'req-f806dcc7-6124-41ad-8fcc-ce7334e0ac39'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:26.295855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:26.298801 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:26.343630 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02344c-6b50-4d3d-ad60-bb84606709e5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4986821ef9f84b99a835e387635aa9a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:26.344336 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02344c-6b50-4d3d-ad60-bb84606709e5 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:26.344561 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02344c-6b50-4d3d-ad60-bb84606709e5 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:26.344831 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02344c-6b50-4d3d-ad60-bb84606709e5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:26.387869 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f806dcc7-6124-41ad-8fcc-ce7334e0ac39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e92e1e1c27241798e336e547580be5e', 'request_id': 'req-f806dcc7-6124-41ad-8fcc-ce7334e0ac39'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:26.395792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f806dcc7-6124-41ad-8fcc-ce7334e0ac39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e92e1e1c27241798e336e547580be5e', 'request_id': 'req-f806dcc7-6124-41ad-8fcc-ce7334e0ac39'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:26.406921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f806dcc7-6124-41ad-8fcc-ce7334e0ac39 admin admin] The token cache is being invalidate because project 7e92e1e1c27241798e336e547580be5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:21:26.407298 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f806dcc7-6124-41ad-8fcc-ce7334e0ac39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:26.408142 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f806dcc7-6124-41ad-8fcc-ce7334e0ac39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:26.409296 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02344c-6b50-4d3d-ad60-bb84606709e5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:26.409828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1185/2389] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:21:26 2022] DELETE /identity/v3/projects/7e92e1e1c27241798e336e547580be5e => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:26.411049 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02344c-6b50-4d3d-ad60-bb84606709e5 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:26.415370 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0503c3-bccf-4d22-9def-b4524283d8d9 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:26.416109 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0503c3-bccf-4d22-9def-b4524283d8d9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:26.417353 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0503c3-bccf-4d22-9def-b4524283d8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:26.436273 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1205/2390] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 10 17:21:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 17:21:26.441746 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-cc0503c3-bccf-4d22-9def-b4524283d8d9 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:21:26.443053 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cc0503c3-bccf-4d22-9def-b4524283d8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:26.506720 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1186/2391] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:21:26.515098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc0643-8982-4a1f-a44c-a2158eba2538 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:26.520885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc0643-8982-4a1f-a44c-a2158eba2538 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:26.521260 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc0643-8982-4a1f-a44c-a2158eba2538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:26.544581 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b9cc0643-8982-4a1f-a44c-a2158eba2538 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:21:26.545867 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b9cc0643-8982-4a1f-a44c-a2158eba2538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:26.558406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1206/2392] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:21:26.562987 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0503c3-bccf-4d22-9def-b4524283d8d9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:26.572316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0503c3-bccf-4d22-9def-b4524283d8d9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:26.577146 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c86ed-ab18-471b-88dd-0593cdd97cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:26.577752 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c86ed-ab18-471b-88dd-0593cdd97cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:26.578027 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c86ed-ab18-471b-88dd-0593cdd97cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:26.578299 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c86ed-ab18-471b-88dd-0593cdd97cf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:26.578977 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c86ed-ab18-471b-88dd-0593cdd97cf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:21:26.579283 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c86ed-ab18-471b-88dd-0593cdd97cf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:26.580998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c86ed-ab18-471b-88dd-0593cdd97cf4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:26.594457 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1187/2393] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:21:26.599179 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc0643-8982-4a1f-a44c-a2158eba2538 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:26.618244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc0643-8982-4a1f-a44c-a2158eba2538 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:26.624066 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0e0e6-c5bf-4a64-ad18-b15231f943d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:26.624861 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0e0e6-c5bf-4a64-ad18-b15231f943d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:26.625141 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0e0e6-c5bf-4a64-ad18-b15231f943d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:26.625412 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0e0e6-c5bf-4a64-ad18-b15231f943d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:26.626277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0e0e6-c5bf-4a64-ad18-b15231f943d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:26.631510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0e0e6-c5bf-4a64-ad18-b15231f943d7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:26.775881 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1207/2394] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:21:26 2022] POST /identity/v3/projects => generated 399 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:21:26.778590 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c86ed-ab18-471b-88dd-0593cdd97cf4 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:26.791339 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c86ed-ab18-471b-88dd-0593cdd97cf4 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:26.942120 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47ea51-b322-44a4-82c1-74421737ae4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:26.944785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47ea51-b322-44a4-82c1-74421737ae4e admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:26.945094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47ea51-b322-44a4-82c1-74421737ae4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:26.946588 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47ea51-b322-44a4-82c1-74421737ae4e admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:26.949310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca47ea51-b322-44a4-82c1-74421737ae4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:26.952731 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca47ea51-b322-44a4-82c1-74421737ae4e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:27.088913 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1188/2395] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:21:26 2022] POST /identity/v3/users => generated 300 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:21:27.093469 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0e0e6-c5bf-4a64-ad18-b15231f943d7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:27.112395 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0e0e6-c5bf-4a64-ad18-b15231f943d7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:27.120354 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b7d574-c6e3-4fb6-a7f6-4a4db7a8c1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:27.120991 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7d574-c6e3-4fb6-a7f6-4a4db7a8c1c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:27.121270 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7d574-c6e3-4fb6-a7f6-4a4db7a8c1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:27.121543 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7d574-c6e3-4fb6-a7f6-4a4db7a8c1c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:27.122260 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b7d574-c6e3-4fb6-a7f6-4a4db7a8c1c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:27.123997 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b7d574-c6e3-4fb6-a7f6-4a4db7a8c1c6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:27.136728 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1208/2396] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:21:27.140711 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47ea51-b322-44a4-82c1-74421737ae4e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:27.153211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47ea51-b322-44a4-82c1-74421737ae4e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:27.161376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1384485-4141-4a48-9d80-b78521d77968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:27.162582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1384485-4141-4a48-9d80-b78521d77968 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:27.163170 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1384485-4141-4a48-9d80-b78521d77968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:27.163613 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1384485-4141-4a48-9d80-b78521d77968 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:27.164792 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1384485-4141-4a48-9d80-b78521d77968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:27.167078 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1384485-4141-4a48-9d80-b78521d77968 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:27.181503 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1189/2397] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:21:27.186345 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b7d574-c6e3-4fb6-a7f6-4a4db7a8c1c6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:27.222743 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b7d574-c6e3-4fb6-a7f6-4a4db7a8c1c6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:27.236020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3284d-8e71-4ae9-a5b4-0aa3311f1fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:27.238117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3284d-8e71-4ae9-a5b4-0aa3311f1fff admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:27.238445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3284d-8e71-4ae9-a5b4-0aa3311f1fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:27.238817 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3284d-8e71-4ae9-a5b4-0aa3311f1fff admin admin] PATH_INFO: `/v3/projects/d8453b01dc8b4344a9f05b97aa48425b/users/96e70a56cce744908c4e53b2d871ea46/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:27.279163 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3284d-8e71-4ae9-a5b4-0aa3311f1fff admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8453b01dc8b4344a9f05b97aa48425b, user_id=96e70a56cce744908c4e53b2d871ea46, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:27.284240 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3284d-8e71-4ae9-a5b4-0aa3311f1fff admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:27.388783 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1209/2398] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:21:27 2022] PUT /identity/v3/projects/d8453b01dc8b4344a9f05b97aa48425b/users/96e70a56cce744908c4e53b2d871ea46/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:27.401044 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff0014-b97b-4ab2-a71d-64e44f0587bd None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:27.401448 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff0014-b97b-4ab2-a71d-64e44f0587bd None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:27.401731 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff0014-b97b-4ab2-a71d-64e44f0587bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:27.449748 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e7ff0014-b97b-4ab2-a71d-64e44f0587bd None None] MFA Rules not processed for user `96e70a56cce744908c4e53b2d871ea46`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:21:27.451812 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e7ff0014-b97b-4ab2-a71d-64e44f0587bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:27.511319 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:27.563416 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1190/2399] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:21:27 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:21:27.569485 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3284d-8e71-4ae9-a5b4-0aa3311f1fff admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:27.654983 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:27.660788 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3284d-8e71-4ae9-a5b4-0aa3311f1fff admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:27.668914 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc03cf-ed87-4c6f-b2d7-d639d8d204cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:27.668914 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc03cf-ed87-4c6f-b2d7-d639d8d204cd admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:27.668914 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc03cf-ed87-4c6f-b2d7-d639d8d204cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:27.668914 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc03cf-ed87-4c6f-b2d7-d639d8d204cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:27.669384 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bc03cf-ed87-4c6f-b2d7-d639d8d204cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:27.670678 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bc03cf-ed87-4c6f-b2d7-d639d8d204cd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:27.798498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1210/2400] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:21:27 2022] POST /identity/v3/projects => generated 399 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:21:27.799038 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff0014-b97b-4ab2-a71d-64e44f0587bd None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:27.824282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff0014-b97b-4ab2-a71d-64e44f0587bd None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:27.968113 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b3651-af00-4f3f-b57d-b3a88b324fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:27.968113 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b3651-af00-4f3f-b57d-b3a88b324fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:27.968113 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b3651-af00-4f3f-b57d-b3a88b324fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:27.968113 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b3651-af00-4f3f-b57d-b3a88b324fd9 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:27.968464 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b3651-af00-4f3f-b57d-b3a88b324fd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:27.969122 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b3651-af00-4f3f-b57d-b3a88b324fd9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:28.093171 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1191/2401] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:21:27 2022] POST /identity/v3/users => generated 306 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:21:28.095185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc03cf-ed87-4c6f-b2d7-d639d8d204cd admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:28.116547 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc03cf-ed87-4c6f-b2d7-d639d8d204cd admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:28.139201 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2892d6f-b50e-4727-9dfb-9e6d50286a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:28.151042 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2892d6f-b50e-4727-9dfb-9e6d50286a88 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:28.151404 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2892d6f-b50e-4727-9dfb-9e6d50286a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:28.151691 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2892d6f-b50e-4727-9dfb-9e6d50286a88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:28.157894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2892d6f-b50e-4727-9dfb-9e6d50286a88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:28.157894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2892d6f-b50e-4727-9dfb-9e6d50286a88 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:28.175719 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1211/2402] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:21:28.176313 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b3651-af00-4f3f-b57d-b3a88b324fd9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:28.190255 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b3651-af00-4f3f-b57d-b3a88b324fd9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:28.199362 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232aec71-e48e-446a-a5b6-76bfb5c5b0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:28.200129 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232aec71-e48e-446a-a5b6-76bfb5c5b0c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:28.200462 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232aec71-e48e-446a-a5b6-76bfb5c5b0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:28.200726 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232aec71-e48e-446a-a5b6-76bfb5c5b0c5 admin admin] PATH_INFO: `/v3/projects/2056312c98f14aaf943dcb6ec2e2bb8a/users/41ad002380b94b65abc1178b8f139497/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:28.239049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232aec71-e48e-446a-a5b6-76bfb5c5b0c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2056312c98f14aaf943dcb6ec2e2bb8a, user_id=41ad002380b94b65abc1178b8f139497, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:28.242117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232aec71-e48e-446a-a5b6-76bfb5c5b0c5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:28.338096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1192/2403] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:21:28 2022] PUT /identity/v3/projects/2056312c98f14aaf943dcb6ec2e2bb8a/users/41ad002380b94b65abc1178b8f139497/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:28.343541 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2892d6f-b50e-4727-9dfb-9e6d50286a88 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:28.360677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2892d6f-b50e-4727-9dfb-9e6d50286a88 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:28.505760 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce908c0a-0cf6-4530-8c65-5254e2048c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:28.506559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce908c0a-0cf6-4530-8c65-5254e2048c66 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:28.509387 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce908c0a-0cf6-4530-8c65-5254e2048c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:28.510241 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce908c0a-0cf6-4530-8c65-5254e2048c66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:28.511384 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce908c0a-0cf6-4530-8c65-5254e2048c66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:28.517729 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce908c0a-0cf6-4530-8c65-5254e2048c66 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:28.547702 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232aec71-e48e-446a-a5b6-76bfb5c5b0c5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:28.561621 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1212/2404] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:21:28.569419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232aec71-e48e-446a-a5b6-76bfb5c5b0c5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:28.586069 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b9c2da-e01d-4cbe-8add-8da06550b32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:28.586885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9c2da-e01d-4cbe-8add-8da06550b32b admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:28.587264 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9c2da-e01d-4cbe-8add-8da06550b32b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:28.587638 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9c2da-e01d-4cbe-8add-8da06550b32b admin admin] PATH_INFO: `/v3/projects/2056312c98f14aaf943dcb6ec2e2bb8a/users/41ad002380b94b65abc1178b8f139497/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:28.590675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b9c2da-e01d-4cbe-8add-8da06550b32b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2056312c98f14aaf943dcb6ec2e2bb8a, user_id=41ad002380b94b65abc1178b8f139497, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:28.598195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b9c2da-e01d-4cbe-8add-8da06550b32b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:28.639648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1193/2405] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:21:28 2022] PUT /identity/v3/projects/2056312c98f14aaf943dcb6ec2e2bb8a/users/41ad002380b94b65abc1178b8f139497/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:21:28.652909 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c383e6f-fb89-4d0f-8525-6e5404123176 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:28.653241 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c383e6f-fb89-4d0f-8525-6e5404123176 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:28.653526 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c383e6f-fb89-4d0f-8525-6e5404123176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:28.709268 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8c383e6f-fb89-4d0f-8525-6e5404123176 None None] MFA Rules not processed for user `41ad002380b94b65abc1178b8f139497`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:21:28.713379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8c383e6f-fb89-4d0f-8525-6e5404123176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:28.864620 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1213/2406] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:21:28 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:21:28.875460 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b9c2da-e01d-4cbe-8add-8da06550b32b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:28.877461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f9b9c2da-e01d-4cbe-8add-8da06550b32b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:28.999335 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b9c2da-e01d-4cbe-8add-8da06550b32b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:29.006449 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533178d-2c70-412c-a646-da6130ad8d32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:29.008528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533178d-2c70-412c-a646-da6130ad8d32 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:29.008992 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533178d-2c70-412c-a646-da6130ad8d32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:29.010128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533178d-2c70-412c-a646-da6130ad8d32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:29.023030 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c533178d-2c70-412c-a646-da6130ad8d32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:29.024472 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c533178d-2c70-412c-a646-da6130ad8d32 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:29.036829 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1194/2407] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:21:28 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:21:29.315174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c383e6f-fb89-4d0f-8525-6e5404123176 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:29.331032 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c383e6f-fb89-4d0f-8525-6e5404123176 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:29.340938 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84540abb-f0a4-4215-9dfe-d7f57b690526 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:29.341564 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84540abb-f0a4-4215-9dfe-d7f57b690526 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:29.341844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84540abb-f0a4-4215-9dfe-d7f57b690526 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:29.342114 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84540abb-f0a4-4215-9dfe-d7f57b690526 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:29.484114 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:29.496666 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84540abb-f0a4-4215-9dfe-d7f57b690526 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:29.498207 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84540abb-f0a4-4215-9dfe-d7f57b690526 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:29.521383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1214/2408] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:21:29 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:21:35.556901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533178d-2c70-412c-a646-da6130ad8d32 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:35.590828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:35.620313 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533178d-2c70-412c-a646-da6130ad8d32 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:35.625092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8d91f-e0e2-4030-986a-abf0f0e07494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:35.625744 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8d91f-e0e2-4030-986a-abf0f0e07494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:35.626022 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8d91f-e0e2-4030-986a-abf0f0e07494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:35.626286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8d91f-e0e2-4030-986a-abf0f0e07494 admin admin] PATH_INFO: `/v3/users/96e70a56cce744908c4e53b2d871ea46` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:35.627428 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde8d91f-e0e2-4030-986a-abf0f0e07494 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96e70a56cce744908c4e53b2d871ea46)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:35.629915 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde8d91f-e0e2-4030-986a-abf0f0e07494 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:35.853208 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cde8d91f-e0e2-4030-986a-abf0f0e07494 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96e70a56cce744908c4e53b2d871ea46'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:35.941507 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cde8d91f-e0e2-4030-986a-abf0f0e07494 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96e70a56cce744908c4e53b2d871ea46'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:35.955229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cde8d91f-e0e2-4030-986a-abf0f0e07494 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96e70a56cce744908c4e53b2d871ea46'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:35.956599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cde8d91f-e0e2-4030-986a-abf0f0e07494 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96e70a56cce744908c4e53b2d871ea46'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:35.961460 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cde8d91f-e0e2-4030-986a-abf0f0e07494 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96e70a56cce744908c4e53b2d871ea46'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:35.971052 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1195/2409] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:21:35 2022] DELETE /identity/v3/users/96e70a56cce744908c4e53b2d871ea46 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:35.975817 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f72b4a-7894-4921-b03e-97b9fab1d08d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:35.976359 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f72b4a-7894-4921-b03e-97b9fab1d08d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:35.976359 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f72b4a-7894-4921-b03e-97b9fab1d08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:35.995436 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-06f72b4a-7894-4921-b03e-97b9fab1d08d None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:21:35.996625 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-06f72b4a-7894-4921-b03e-97b9fab1d08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:36.055172 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1215/2410] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:21:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:21:36.064892 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8d91f-e0e2-4030-986a-abf0f0e07494 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:36.067086 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cde8d91f-e0e2-4030-986a-abf0f0e07494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:36.123384 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8d91f-e0e2-4030-986a-abf0f0e07494 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:36.127004 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12655b-1d5b-4411-a332-91b5ef017e20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:36.127608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12655b-1d5b-4411-a332-91b5ef017e20 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:36.128414 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12655b-1d5b-4411-a332-91b5ef017e20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:36.128719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12655b-1d5b-4411-a332-91b5ef017e20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:36.138129 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce12655b-1d5b-4411-a332-91b5ef017e20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:36.139599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce12655b-1d5b-4411-a332-91b5ef017e20 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:36.149536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1196/2411] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:21:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:21:36.920492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f72b4a-7894-4921-b03e-97b9fab1d08d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:36.921920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-06f72b4a-7894-4921-b03e-97b9fab1d08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:36.937898 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f72b4a-7894-4921-b03e-97b9fab1d08d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:36.955368 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbbc5d-672d-45ec-abc7-3121d43011e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:36.957265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbbc5d-672d-45ec-abc7-3121d43011e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:36.957900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbbc5d-672d-45ec-abc7-3121d43011e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:36.958415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbbc5d-672d-45ec-abc7-3121d43011e6 admin admin] PATH_INFO: `/v3/projects/d8453b01dc8b4344a9f05b97aa48425b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:36.959916 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbbc5d-672d-45ec-abc7-3121d43011e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8453b01dc8b4344a9f05b97aa48425b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:36.962057 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbbc5d-672d-45ec-abc7-3121d43011e6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:37.091761 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cfdbbc5d-672d-45ec-abc7-3121d43011e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8453b01dc8b4344a9f05b97aa48425b', 'request_id': 'req-cfdbbc5d-672d-45ec-abc7-3121d43011e6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:37.098886 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cfdbbc5d-672d-45ec-abc7-3121d43011e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8453b01dc8b4344a9f05b97aa48425b', 'request_id': 'req-cfdbbc5d-672d-45ec-abc7-3121d43011e6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:37.181945 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cfdbbc5d-672d-45ec-abc7-3121d43011e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8453b01dc8b4344a9f05b97aa48425b', 'request_id': 'req-cfdbbc5d-672d-45ec-abc7-3121d43011e6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:37.193345 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cfdbbc5d-672d-45ec-abc7-3121d43011e6 admin admin] The token cache is being invalidate because project d8453b01dc8b4344a9f05b97aa48425b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:21:37.204279 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cfdbbc5d-672d-45ec-abc7-3121d43011e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:37.204279 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cfdbbc5d-672d-45ec-abc7-3121d43011e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:37.204279 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1216/2412] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:21:36 2022] DELETE /identity/v3/projects/d8453b01dc8b4344a9f05b97aa48425b => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:37.204587 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12655b-1d5b-4411-a332-91b5ef017e20 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:37.210570 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12655b-1d5b-4411-a332-91b5ef017e20 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:37.267540 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:37.279711 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d9668-e9ec-4a97-9ff7-38175aab9b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:37.284716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d9668-e9ec-4a97-9ff7-38175aab9b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:37.284716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d9668-e9ec-4a97-9ff7-38175aab9b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:37.284716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d9668-e9ec-4a97-9ff7-38175aab9b3c admin admin] PATH_INFO: `/v3/users/41ad002380b94b65abc1178b8f139497` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:37.284716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d9668-e9ec-4a97-9ff7-38175aab9b3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=41ad002380b94b65abc1178b8f139497)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:37.284716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d9668-e9ec-4a97-9ff7-38175aab9b3c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:37.447025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-164d9668-e9ec-4a97-9ff7-38175aab9b3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41ad002380b94b65abc1178b8f139497'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:37.507811 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-164d9668-e9ec-4a97-9ff7-38175aab9b3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41ad002380b94b65abc1178b8f139497'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:37.523767 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-164d9668-e9ec-4a97-9ff7-38175aab9b3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41ad002380b94b65abc1178b8f139497'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:37.524510 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-164d9668-e9ec-4a97-9ff7-38175aab9b3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41ad002380b94b65abc1178b8f139497'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:37.525131 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-164d9668-e9ec-4a97-9ff7-38175aab9b3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41ad002380b94b65abc1178b8f139497'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:37.541340 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1197/2413] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:21:37 2022] DELETE /identity/v3/users/41ad002380b94b65abc1178b8f139497 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:38.019793 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbbc5d-672d-45ec-abc7-3121d43011e6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:38.021442 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cfdbbc5d-672d-45ec-abc7-3121d43011e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:38.044318 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbbc5d-672d-45ec-abc7-3121d43011e6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:38.140435 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd93d4-8ffc-4a44-a28a-bb7decc61c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:38.140957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd93d4-8ffc-4a44-a28a-bb7decc61c79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:38.141217 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd93d4-8ffc-4a44-a28a-bb7decc61c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:38.141548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd93d4-8ffc-4a44-a28a-bb7decc61c79 admin admin] PATH_INFO: `/v3/projects/2056312c98f14aaf943dcb6ec2e2bb8a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:38.142786 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbd93d4-8ffc-4a44-a28a-bb7decc61c79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2056312c98f14aaf943dcb6ec2e2bb8a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:38.145326 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbd93d4-8ffc-4a44-a28a-bb7decc61c79 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:38.165175 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:38.259019 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7cbd93d4-8ffc-4a44-a28a-bb7decc61c79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2056312c98f14aaf943dcb6ec2e2bb8a', 'request_id': 'req-7cbd93d4-8ffc-4a44-a28a-bb7decc61c79'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:38.266823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7cbd93d4-8ffc-4a44-a28a-bb7decc61c79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2056312c98f14aaf943dcb6ec2e2bb8a', 'request_id': 'req-7cbd93d4-8ffc-4a44-a28a-bb7decc61c79'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:38.382324 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7cbd93d4-8ffc-4a44-a28a-bb7decc61c79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2056312c98f14aaf943dcb6ec2e2bb8a', 'request_id': 'req-7cbd93d4-8ffc-4a44-a28a-bb7decc61c79'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:38.394718 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7cbd93d4-8ffc-4a44-a28a-bb7decc61c79 admin admin] The token cache is being invalidate because project 2056312c98f14aaf943dcb6ec2e2bb8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:21:38.395014 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7cbd93d4-8ffc-4a44-a28a-bb7decc61c79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:38.395666 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7cbd93d4-8ffc-4a44-a28a-bb7decc61c79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:38.397205 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1217/2414] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:21:38 2022] DELETE /identity/v3/projects/2056312c98f14aaf943dcb6ec2e2bb8a => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:38.402060 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6edbd-3c39-471c-a032-d7b7f0870412 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:38.402633 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6edbd-3c39-471c-a032-d7b7f0870412 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:38.403271 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6edbd-3c39-471c-a032-d7b7f0870412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:38.425405 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3fb6edbd-3c39-471c-a032-d7b7f0870412 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:21:38.429642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3fb6edbd-3c39-471c-a032-d7b7f0870412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:38.494176 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1198/2415] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:21:38.502013 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225366a-c45c-429a-8e73-b00a454a083f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:38.502298 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225366a-c45c-429a-8e73-b00a454a083f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:38.502630 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225366a-c45c-429a-8e73-b00a454a083f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:38.524010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e225366a-c45c-429a-8e73-b00a454a083f None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:21:38.525126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e225366a-c45c-429a-8e73-b00a454a083f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:38.533152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1218/2416] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:21:38.537572 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6edbd-3c39-471c-a032-d7b7f0870412 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:38.547398 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6edbd-3c39-471c-a032-d7b7f0870412 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:38.552750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee950c7-de51-4872-8f38-a7b5f1d1b012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:38.553353 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee950c7-de51-4872-8f38-a7b5f1d1b012 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:38.553626 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee950c7-de51-4872-8f38-a7b5f1d1b012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:38.553895 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee950c7-de51-4872-8f38-a7b5f1d1b012 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:38.554609 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee950c7-de51-4872-8f38-a7b5f1d1b012 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:21:38.554989 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee950c7-de51-4872-8f38-a7b5f1d1b012 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:38.556547 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee950c7-de51-4872-8f38-a7b5f1d1b012 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:38.571108 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1199/2417] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:21:38.575582 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225366a-c45c-429a-8e73-b00a454a083f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:38.585819 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225366a-c45c-429a-8e73-b00a454a083f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:38.590960 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d58e3-d196-4060-99cc-450cf01a6bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:38.591659 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d58e3-d196-4060-99cc-450cf01a6bdd admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:38.591989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d58e3-d196-4060-99cc-450cf01a6bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:38.592395 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d58e3-d196-4060-99cc-450cf01a6bdd admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:38.593116 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d58e3-d196-4060-99cc-450cf01a6bdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:38.595197 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d58e3-d196-4060-99cc-450cf01a6bdd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:38.722296 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1219/2418] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:21:38 2022] POST /identity/v3/projects => generated 389 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:21:38.726140 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee950c7-de51-4872-8f38-a7b5f1d1b012 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:38.736670 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee950c7-de51-4872-8f38-a7b5f1d1b012 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:38.806301 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a9b0c-84d6-4f51-8f4d-bc13510411ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:38.806861 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a9b0c-84d6-4f51-8f4d-bc13510411ee admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:38.807214 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a9b0c-84d6-4f51-8f4d-bc13510411ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:38.807488 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a9b0c-84d6-4f51-8f4d-bc13510411ee admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:38.808112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a9b0c-84d6-4f51-8f4d-bc13510411ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:38.810089 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a9b0c-84d6-4f51-8f4d-bc13510411ee admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:38.818172 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:39.863612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1200/2419] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:21:38 2022] POST /identity/v3/users => generated 295 bytes in 1137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:21:39.867672 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d58e3-d196-4060-99cc-450cf01a6bdd admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:39.878198 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d58e3-d196-4060-99cc-450cf01a6bdd admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:39.883159 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7926903-9249-4baa-8fc5-6c9c6c955ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:39.883755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7926903-9249-4baa-8fc5-6c9c6c955ade admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:39.884031 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7926903-9249-4baa-8fc5-6c9c6c955ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:39.884300 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7926903-9249-4baa-8fc5-6c9c6c955ade admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:39.884935 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7926903-9249-4baa-8fc5-6c9c6c955ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:39.886577 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7926903-9249-4baa-8fc5-6c9c6c955ade admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:39.905819 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1220/2420] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:21:39.906230 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a9b0c-84d6-4f51-8f4d-bc13510411ee admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:39.924409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a9b0c-84d6-4f51-8f4d-bc13510411ee admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:39.930240 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6b054-e07f-49ef-adf9-1e8f7002e112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:39.932608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6b054-e07f-49ef-adf9-1e8f7002e112 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:39.932608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6b054-e07f-49ef-adf9-1e8f7002e112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:39.932608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6b054-e07f-49ef-adf9-1e8f7002e112 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:39.932608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba6b054-e07f-49ef-adf9-1e8f7002e112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:39.933754 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba6b054-e07f-49ef-adf9-1e8f7002e112 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:39.950653 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1201/2421] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:21:39.953554 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7926903-9249-4baa-8fc5-6c9c6c955ade admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:39.965851 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7926903-9249-4baa-8fc5-6c9c6c955ade admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:39.970525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899979e-67f3-4200-b05e-957d67145e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:39.971165 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899979e-67f3-4200-b05e-957d67145e32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:39.971438 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899979e-67f3-4200-b05e-957d67145e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:39.971701 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899979e-67f3-4200-b05e-957d67145e32 admin admin] PATH_INFO: `/v3/projects/7969f2e3c6544ea3a27a3809b953dddd/users/028897faa9924de5b5298e259e6c3ed4/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:39.990789 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4899979e-67f3-4200-b05e-957d67145e32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7969f2e3c6544ea3a27a3809b953dddd, user_id=028897faa9924de5b5298e259e6c3ed4, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:39.993893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4899979e-67f3-4200-b05e-957d67145e32 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:40.131419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1221/2422] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:21:39 2022] PUT /identity/v3/projects/7969f2e3c6544ea3a27a3809b953dddd/users/028897faa9924de5b5298e259e6c3ed4/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:40.143206 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ddc7b-7253-4fae-ad1f-c1be8b860ca6 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:40.143485 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ddc7b-7253-4fae-ad1f-c1be8b860ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:40.143752 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ddc7b-7253-4fae-ad1f-c1be8b860ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:40.173094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-aa9ddc7b-7253-4fae-ad1f-c1be8b860ca6 None None] MFA Rules not processed for user `028897faa9924de5b5298e259e6c3ed4`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:21:40.174304 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-aa9ddc7b-7253-4fae-ad1f-c1be8b860ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:40.239434 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1202/2423] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:21:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:21:40.244085 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899979e-67f3-4200-b05e-957d67145e32 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:40.315422 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899979e-67f3-4200-b05e-957d67145e32 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:40.321355 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb7d1c-aff3-4045-a92b-a59698504319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:40.321961 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb7d1c-aff3-4045-a92b-a59698504319 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:40.322235 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb7d1c-aff3-4045-a92b-a59698504319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:40.322534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb7d1c-aff3-4045-a92b-a59698504319 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:40.323174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb7d1c-aff3-4045-a92b-a59698504319 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:40.325073 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb7d1c-aff3-4045-a92b-a59698504319 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:40.446277 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1222/2424] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:21:40 2022] POST /identity/v3/projects => generated 389 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:21:40.447517 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ddc7b-7253-4fae-ad1f-c1be8b860ca6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:40.461327 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ddc7b-7253-4fae-ad1f-c1be8b860ca6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:40.526648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd74f05-814f-4666-9244-2c38b3d2401b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:40.527252 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd74f05-814f-4666-9244-2c38b3d2401b admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:40.527528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd74f05-814f-4666-9244-2c38b3d2401b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:40.527797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd74f05-814f-4666-9244-2c38b3d2401b admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:40.528411 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd74f05-814f-4666-9244-2c38b3d2401b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:40.530370 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd74f05-814f-4666-9244-2c38b3d2401b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:40.651812 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1203/2425] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:21:40 2022] POST /identity/v3/users => generated 301 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:21:40.655915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb7d1c-aff3-4045-a92b-a59698504319 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:40.666076 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb7d1c-aff3-4045-a92b-a59698504319 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:40.670597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e887a2a-1b20-488c-bae6-058d9d9e5aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:40.671205 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e887a2a-1b20-488c-bae6-058d9d9e5aec admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:40.671477 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e887a2a-1b20-488c-bae6-058d9d9e5aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:40.671740 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e887a2a-1b20-488c-bae6-058d9d9e5aec admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:40.672372 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e887a2a-1b20-488c-bae6-058d9d9e5aec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:40.673950 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e887a2a-1b20-488c-bae6-058d9d9e5aec admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:40.685930 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1223/2426] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:21:40.690266 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd74f05-814f-4666-9244-2c38b3d2401b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:40.701489 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd74f05-814f-4666-9244-2c38b3d2401b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:40.706279 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf3e36-4354-4529-bc91-1176696e39e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:40.707638 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf3e36-4354-4529-bc91-1176696e39e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:40.707975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf3e36-4354-4529-bc91-1176696e39e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:40.708251 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf3e36-4354-4529-bc91-1176696e39e5 admin admin] PATH_INFO: `/v3/projects/ebc81a54e1644904a3069726948ffbd8/users/d07fcd7b53e349cfb37b8ec08f3c6f2d/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:40.723178 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf3e36-4354-4529-bc91-1176696e39e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebc81a54e1644904a3069726948ffbd8, user_id=d07fcd7b53e349cfb37b8ec08f3c6f2d, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:40.726091 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf3e36-4354-4529-bc91-1176696e39e5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:40.799123 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1204/2427] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:21:40 2022] PUT /identity/v3/projects/ebc81a54e1644904a3069726948ffbd8/users/d07fcd7b53e349cfb37b8ec08f3c6f2d/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:40.803750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e887a2a-1b20-488c-bae6-058d9d9e5aec admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:40.813846 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e887a2a-1b20-488c-bae6-058d9d9e5aec admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:40.827302 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:40.870953 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4541f-724c-4bd0-8c37-930b93ea4994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:40.871602 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4541f-724c-4bd0-8c37-930b93ea4994 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:40.871846 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4541f-724c-4bd0-8c37-930b93ea4994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:40.872118 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4541f-724c-4bd0-8c37-930b93ea4994 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:40.872785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf4541f-724c-4bd0-8c37-930b93ea4994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:40.874400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf4541f-724c-4bd0-8c37-930b93ea4994 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:40.885625 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1224/2428] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:21:40.889837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf3e36-4354-4529-bc91-1176696e39e5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:40.901199 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf3e36-4354-4529-bc91-1176696e39e5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:40.906031 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a04af-7420-49a6-b214-67c099633773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:40.906645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a04af-7420-49a6-b214-67c099633773 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:40.906916 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a04af-7420-49a6-b214-67c099633773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:40.907433 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a04af-7420-49a6-b214-67c099633773 admin admin] PATH_INFO: `/v3/projects/ebc81a54e1644904a3069726948ffbd8/users/d07fcd7b53e349cfb37b8ec08f3c6f2d/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:40.909078 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a04af-7420-49a6-b214-67c099633773 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebc81a54e1644904a3069726948ffbd8, user_id=d07fcd7b53e349cfb37b8ec08f3c6f2d, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:40.912571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a04af-7420-49a6-b214-67c099633773 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:40.923057 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1205/2429] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:21:40 2022] PUT /identity/v3/projects/ebc81a54e1644904a3069726948ffbd8/users/d07fcd7b53e349cfb37b8ec08f3c6f2d/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:40.933404 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4529d-90fd-478e-b1c8-3b20ac4a657e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:40.933693 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4529d-90fd-478e-b1c8-3b20ac4a657e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:40.933969 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4529d-90fd-478e-b1c8-3b20ac4a657e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:40.961768 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a6a4529d-90fd-478e-b1c8-3b20ac4a657e None None] MFA Rules not processed for user `d07fcd7b53e349cfb37b8ec08f3c6f2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:21:40.962997 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a6a4529d-90fd-478e-b1c8-3b20ac4a657e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:41.030011 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1225/2430] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:21:40 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:21:41.041684 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a04af-7420-49a6-b214-67c099633773 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:41.043454 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c64a04af-7420-49a6-b214-67c099633773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:41.065559 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:41.100573 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a04af-7420-49a6-b214-67c099633773 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:41.104927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713898fa-695a-46de-981b-87d304338e93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:41.105523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713898fa-695a-46de-981b-87d304338e93 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:41.105793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713898fa-695a-46de-981b-87d304338e93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:41.106059 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713898fa-695a-46de-981b-87d304338e93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:41.115098 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713898fa-695a-46de-981b-87d304338e93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:41.119693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713898fa-695a-46de-981b-87d304338e93 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:41.125564 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1206/2431] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:21:41 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:21:41.268769 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4529d-90fd-478e-b1c8-3b20ac4a657e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:41.352902 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4529d-90fd-478e-b1c8-3b20ac4a657e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:41.357611 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee543d-ece8-4765-aa5c-7236781d4a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:41.371624 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee543d-ece8-4765-aa5c-7236781d4a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:41.371624 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee543d-ece8-4765-aa5c-7236781d4a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:41.371624 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee543d-ece8-4765-aa5c-7236781d4a27 admin admin] PATH_INFO: `/v3/users/028897faa9924de5b5298e259e6c3ed4` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:41.371624 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee543d-ece8-4765-aa5c-7236781d4a27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=028897faa9924de5b5298e259e6c3ed4)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:41.371624 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee543d-ece8-4765-aa5c-7236781d4a27 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:41.580956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:41.586725 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5eee543d-ece8-4765-aa5c-7236781d4a27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '028897faa9924de5b5298e259e6c3ed4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:41.673462 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5eee543d-ece8-4765-aa5c-7236781d4a27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '028897faa9924de5b5298e259e6c3ed4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:41.681691 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5eee543d-ece8-4765-aa5c-7236781d4a27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '028897faa9924de5b5298e259e6c3ed4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:41.695500 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5eee543d-ece8-4765-aa5c-7236781d4a27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '028897faa9924de5b5298e259e6c3ed4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:41.696352 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5eee543d-ece8-4765-aa5c-7236781d4a27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '028897faa9924de5b5298e259e6c3ed4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:41.702309 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1226/2432] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:21:41 2022] DELETE /identity/v3/users/028897faa9924de5b5298e259e6c3ed4 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:41.712019 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b627bf-8872-4a63-bec8-d1c3bb786222 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:41.712229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b627bf-8872-4a63-bec8-d1c3bb786222 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:41.719126 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b627bf-8872-4a63-bec8-d1c3bb786222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:41.731404 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b1b627bf-8872-4a63-bec8-d1c3bb786222 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:21:41.732603 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b1b627bf-8872-4a63-bec8-d1c3bb786222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:41.792087 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1207/2433] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:21:41.802454 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee543d-ece8-4765-aa5c-7236781d4a27 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:41.803871 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5eee543d-ece8-4765-aa5c-7236781d4a27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:41.865100 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee543d-ece8-4765-aa5c-7236781d4a27 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:41.868717 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717dcc16-31ee-4a13-ba3a-3d1bd361aa65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:41.869318 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dcc16-31ee-4a13-ba3a-3d1bd361aa65 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:41.869590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dcc16-31ee-4a13-ba3a-3d1bd361aa65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:41.869853 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dcc16-31ee-4a13-ba3a-3d1bd361aa65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:41.879014 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717dcc16-31ee-4a13-ba3a-3d1bd361aa65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:41.880552 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717dcc16-31ee-4a13-ba3a-3d1bd361aa65 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:41.890569 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1227/2434] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:21:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:21:42.373027 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b627bf-8872-4a63-bec8-d1c3bb786222 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:42.375569 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b1b627bf-8872-4a63-bec8-d1c3bb786222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:42.394835 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b627bf-8872-4a63-bec8-d1c3bb786222 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:42.400894 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc122ad-1e3a-4fa3-9a50-9ed90b5ade76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:42.401757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc122ad-1e3a-4fa3-9a50-9ed90b5ade76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:42.402144 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc122ad-1e3a-4fa3-9a50-9ed90b5ade76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:42.402491 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc122ad-1e3a-4fa3-9a50-9ed90b5ade76 admin admin] PATH_INFO: `/v3/projects/7969f2e3c6544ea3a27a3809b953dddd` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:42.406443 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc122ad-1e3a-4fa3-9a50-9ed90b5ade76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7969f2e3c6544ea3a27a3809b953dddd)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:42.419516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc122ad-1e3a-4fa3-9a50-9ed90b5ade76 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:42.542541 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-acc122ad-1e3a-4fa3-9a50-9ed90b5ade76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7969f2e3c6544ea3a27a3809b953dddd', 'request_id': 'req-acc122ad-1e3a-4fa3-9a50-9ed90b5ade76'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:42.633195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-acc122ad-1e3a-4fa3-9a50-9ed90b5ade76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7969f2e3c6544ea3a27a3809b953dddd', 'request_id': 'req-acc122ad-1e3a-4fa3-9a50-9ed90b5ade76'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:42.640141 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-acc122ad-1e3a-4fa3-9a50-9ed90b5ade76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7969f2e3c6544ea3a27a3809b953dddd', 'request_id': 'req-acc122ad-1e3a-4fa3-9a50-9ed90b5ade76'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:42.653510 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-acc122ad-1e3a-4fa3-9a50-9ed90b5ade76 admin admin] The token cache is being invalidate because project 7969f2e3c6544ea3a27a3809b953dddd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:21:42.653869 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-acc122ad-1e3a-4fa3-9a50-9ed90b5ade76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:42.654650 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-acc122ad-1e3a-4fa3-9a50-9ed90b5ade76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:42.659354 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1208/2435] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:21:42 2022] DELETE /identity/v3/projects/7969f2e3c6544ea3a27a3809b953dddd => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:42.661282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717dcc16-31ee-4a13-ba3a-3d1bd361aa65 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:42.671568 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717dcc16-31ee-4a13-ba3a-3d1bd361aa65 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:42.759086 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c71a70-f08b-4d7f-8bd4-ad2338e6f795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:42.759856 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c71a70-f08b-4d7f-8bd4-ad2338e6f795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:42.760200 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c71a70-f08b-4d7f-8bd4-ad2338e6f795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:42.760541 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c71a70-f08b-4d7f-8bd4-ad2338e6f795 admin admin] PATH_INFO: `/v3/users/d07fcd7b53e349cfb37b8ec08f3c6f2d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:42.761749 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c71a70-f08b-4d7f-8bd4-ad2338e6f795 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d07fcd7b53e349cfb37b8ec08f3c6f2d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:42.763934 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c71a70-f08b-4d7f-8bd4-ad2338e6f795 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:43.083563 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f9c71a70-f08b-4d7f-8bd4-ad2338e6f795 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd07fcd7b53e349cfb37b8ec08f3c6f2d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:43.203536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f9c71a70-f08b-4d7f-8bd4-ad2338e6f795 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd07fcd7b53e349cfb37b8ec08f3c6f2d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:43.211231 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:43.213287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f9c71a70-f08b-4d7f-8bd4-ad2338e6f795 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd07fcd7b53e349cfb37b8ec08f3c6f2d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:43.227670 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f9c71a70-f08b-4d7f-8bd4-ad2338e6f795 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd07fcd7b53e349cfb37b8ec08f3c6f2d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:43.228940 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f9c71a70-f08b-4d7f-8bd4-ad2338e6f795 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd07fcd7b53e349cfb37b8ec08f3c6f2d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:43.235798 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1228/2436] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:21:42 2022] DELETE /identity/v3/users/d07fcd7b53e349cfb37b8ec08f3c6f2d => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:43.346940 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34f41c-ee48-400b-adfd-9fcdf1a09ba4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:43.346940 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34f41c-ee48-400b-adfd-9fcdf1a09ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:43.346940 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34f41c-ee48-400b-adfd-9fcdf1a09ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:43.377738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5e34f41c-ee48-400b-adfd-9fcdf1a09ba4 None None] MFA Rules not processed for user `4f9e1767ec90435f80248a2930081265`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:21:43.379314 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5e34f41c-ee48-400b-adfd-9fcdf1a09ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:43.425905 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:43.439339 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1209/2437] 10.222.0.45 () {58 vars in 967 bytes} [Wed Aug 10 17:21:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:21:43.452295 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c71a70-f08b-4d7f-8bd4-ad2338e6f795 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:43.453700 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f9c71a70-f08b-4d7f-8bd4-ad2338e6f795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:43.531047 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c71a70-f08b-4d7f-8bd4-ad2338e6f795 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:43.538519 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d14d0-7c03-420d-8ad2-cd3ad80c2e48 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4986821ef9f84b99a835e387635aa9a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:43.539182 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d14d0-7c03-420d-8ad2-cd3ad80c2e48 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:43.539464 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d14d0-7c03-420d-8ad2-cd3ad80c2e48 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:43.539735 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d14d0-7c03-420d-8ad2-cd3ad80c2e48 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:43.550432 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d14d0-7c03-420d-8ad2-cd3ad80c2e48 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:43.553835 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d14d0-7c03-420d-8ad2-cd3ad80c2e48 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:43.569711 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1229/2438] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 10 17:21:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 17:21:43.869397 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34f41c-ee48-400b-adfd-9fcdf1a09ba4 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:43.871357 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5e34f41c-ee48-400b-adfd-9fcdf1a09ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:43.953871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34f41c-ee48-400b-adfd-9fcdf1a09ba4 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:43.959793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd87cf-a933-471f-9c87-ff92374156c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:43.960445 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd87cf-a933-471f-9c87-ff92374156c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:43.960720 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd87cf-a933-471f-9c87-ff92374156c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:43.961215 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd87cf-a933-471f-9c87-ff92374156c4 admin admin] PATH_INFO: `/v3/projects/ebc81a54e1644904a3069726948ffbd8` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:43.962814 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dd87cf-a933-471f-9c87-ff92374156c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebc81a54e1644904a3069726948ffbd8)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:43.965400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dd87cf-a933-471f-9c87-ff92374156c4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:44.180677 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-68dd87cf-a933-471f-9c87-ff92374156c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebc81a54e1644904a3069726948ffbd8', 'request_id': 'req-68dd87cf-a933-471f-9c87-ff92374156c4'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:44.296367 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-68dd87cf-a933-471f-9c87-ff92374156c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebc81a54e1644904a3069726948ffbd8', 'request_id': 'req-68dd87cf-a933-471f-9c87-ff92374156c4'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:44.307622 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-68dd87cf-a933-471f-9c87-ff92374156c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebc81a54e1644904a3069726948ffbd8', 'request_id': 'req-68dd87cf-a933-471f-9c87-ff92374156c4'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:44.316850 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-68dd87cf-a933-471f-9c87-ff92374156c4 admin admin] The token cache is being invalidate because project ebc81a54e1644904a3069726948ffbd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:21:44.317090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-68dd87cf-a933-471f-9c87-ff92374156c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:44.317744 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-68dd87cf-a933-471f-9c87-ff92374156c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:44.319258 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1210/2439] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:21:43 2022] DELETE /identity/v3/projects/ebc81a54e1644904a3069726948ffbd8 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:44.324128 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf27d2a-758d-47a1-9f48-aa316d4394f2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:44.324599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf27d2a-758d-47a1-9f48-aa316d4394f2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:44.324721 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf27d2a-758d-47a1-9f48-aa316d4394f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:44.350956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bbf27d2a-758d-47a1-9f48-aa316d4394f2 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:21:44.352215 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bbf27d2a-758d-47a1-9f48-aa316d4394f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:44.432995 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1230/2440] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:21:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:21:44.443101 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a98a5-7664-42ca-b07c-1d698fea3115 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:44.443660 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a98a5-7664-42ca-b07c-1d698fea3115 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:44.443660 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a98a5-7664-42ca-b07c-1d698fea3115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:44.464042 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d08a98a5-7664-42ca-b07c-1d698fea3115 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:21:44.472207 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d08a98a5-7664-42ca-b07c-1d698fea3115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:44.473527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1211/2441] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:21:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:21:44.477956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf27d2a-758d-47a1-9f48-aa316d4394f2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:44.487703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf27d2a-758d-47a1-9f48-aa316d4394f2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:44.492952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f342d1-f6e3-4d7d-a951-b953e1023f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:44.493563 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f342d1-f6e3-4d7d-a951-b953e1023f05 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:44.493841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f342d1-f6e3-4d7d-a951-b953e1023f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:44.497469 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f342d1-f6e3-4d7d-a951-b953e1023f05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:44.497878 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f342d1-f6e3-4d7d-a951-b953e1023f05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:21:44.498205 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f342d1-f6e3-4d7d-a951-b953e1023f05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:44.500202 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f342d1-f6e3-4d7d-a951-b953e1023f05 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:44.519540 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1231/2442] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:21:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:21:44.521227 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a98a5-7664-42ca-b07c-1d698fea3115 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:44.539808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a98a5-7664-42ca-b07c-1d698fea3115 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:44.544916 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f342d1-f6e3-4d7d-a951-b953e1023f05 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:44.545179 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b7fb7-ae49-45b1-b552-6f42c32c227d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:44.546042 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b7fb7-ae49-45b1-b552-6f42c32c227d admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:44.546626 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b7fb7-ae49-45b1-b552-6f42c32c227d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:44.546887 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-05f342d1-f6e3-4d7d-a951-b953e1023f05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:44.547298 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b7fb7-ae49-45b1-b552-6f42c32c227d admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:44.548130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719b7fb7-ae49-45b1-b552-6f42c32c227d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:44.550182 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719b7fb7-ae49-45b1-b552-6f42c32c227d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:44.612385 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f342d1-f6e3-4d7d-a951-b953e1023f05 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:44.617673 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69180d2e-ac8d-49a5-b568-7d54b94e4ae1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:44.618483 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69180d2e-ac8d-49a5-b568-7d54b94e4ae1 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:44.618836 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69180d2e-ac8d-49a5-b568-7d54b94e4ae1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:44.619115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69180d2e-ac8d-49a5-b568-7d54b94e4ae1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:44.620872 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-69180d2e-ac8d-49a5-b568-7d54b94e4ae1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:44.635170 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69180d2e-ac8d-49a5-b568-7d54b94e4ae1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:44.636640 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69180d2e-ac8d-49a5-b568-7d54b94e4ae1 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:44.640750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:44.647911 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1232/2443] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:21:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:21:44.672297 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1212/2444] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:21:44 2022] POST /identity/v3/projects => generated 387 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:21:44.673559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69180d2e-ac8d-49a5-b568-7d54b94e4ae1 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:44.686812 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69180d2e-ac8d-49a5-b568-7d54b94e4ae1 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:44.761845 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1257f-e532-4184-94e0-881716a89dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:44.766242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1257f-e532-4184-94e0-881716a89dd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:44.766242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1257f-e532-4184-94e0-881716a89dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:44.766242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1257f-e532-4184-94e0-881716a89dd9 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:44.766242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1257f-e532-4184-94e0-881716a89dd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:44.766242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1257f-e532-4184-94e0-881716a89dd9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:44.868486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1233/2445] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:21:44 2022] POST /identity/v3/users => generated 294 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:21:44.879064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b7fb7-ae49-45b1-b552-6f42c32c227d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:44.890021 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b7fb7-ae49-45b1-b552-6f42c32c227d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:44.896323 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4dd06a-643d-49e0-ad6f-6777d49de6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:44.896969 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4dd06a-643d-49e0-ad6f-6777d49de6a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:44.897250 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4dd06a-643d-49e0-ad6f-6777d49de6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:44.897543 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4dd06a-643d-49e0-ad6f-6777d49de6a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:44.898151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4dd06a-643d-49e0-ad6f-6777d49de6a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:44.899991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4dd06a-643d-49e0-ad6f-6777d49de6a0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:44.914473 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1213/2446] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:21:44.915585 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1257f-e532-4184-94e0-881716a89dd9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:44.926736 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1257f-e532-4184-94e0-881716a89dd9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:44.931801 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84477f8-3502-4ed8-a104-aa5f71148a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:44.932413 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84477f8-3502-4ed8-a104-aa5f71148a69 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:44.932692 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84477f8-3502-4ed8-a104-aa5f71148a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:44.932965 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84477f8-3502-4ed8-a104-aa5f71148a69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:44.933607 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84477f8-3502-4ed8-a104-aa5f71148a69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:44.935261 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84477f8-3502-4ed8-a104-aa5f71148a69 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:44.948154 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1234/2447] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:21:44.960076 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4dd06a-643d-49e0-ad6f-6777d49de6a0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:44.970310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4dd06a-643d-49e0-ad6f-6777d49de6a0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:44.975422 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2182d-fb69-4384-b602-992cd7f6e30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:44.976114 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2182d-fb69-4384-b602-992cd7f6e30f admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:44.976391 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2182d-fb69-4384-b602-992cd7f6e30f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:44.976662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2182d-fb69-4384-b602-992cd7f6e30f admin admin] PATH_INFO: `/v3/projects/8da1a18135cf4dcba39c05532dbd6a3a/users/c76daa5ae57646999f4d832a4014a39d/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:44.993123 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce2182d-fb69-4384-b602-992cd7f6e30f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8da1a18135cf4dcba39c05532dbd6a3a, user_id=c76daa5ae57646999f4d832a4014a39d, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:44.995918 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce2182d-fb69-4384-b602-992cd7f6e30f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:45.108841 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1214/2448] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:21:44 2022] PUT /identity/v3/projects/8da1a18135cf4dcba39c05532dbd6a3a/users/c76daa5ae57646999f4d832a4014a39d/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:45.119542 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f2cb4-0397-43c5-bf96-67cddb3c201d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:45.119831 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f2cb4-0397-43c5-bf96-67cddb3c201d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:45.120106 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f2cb4-0397-43c5-bf96-67cddb3c201d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:45.153505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ed8f2cb4-0397-43c5-bf96-67cddb3c201d None None] MFA Rules not processed for user `c76daa5ae57646999f4d832a4014a39d`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:21:45.154883 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ed8f2cb4-0397-43c5-bf96-67cddb3c201d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:45.235113 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1235/2449] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:21:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:21:45.240220 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2182d-fb69-4384-b602-992cd7f6e30f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:45.270901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:45.326519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2182d-fb69-4384-b602-992cd7f6e30f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:45.333011 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc16329-9b3d-403c-a58c-f3e23379f74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:45.333722 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc16329-9b3d-403c-a58c-f3e23379f74e admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:45.334076 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc16329-9b3d-403c-a58c-f3e23379f74e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:45.334524 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc16329-9b3d-403c-a58c-f3e23379f74e admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:45.335169 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc16329-9b3d-403c-a58c-f3e23379f74e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:45.337219 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc16329-9b3d-403c-a58c-f3e23379f74e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:45.434782 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1215/2450] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:21:45 2022] POST /identity/v3/projects => generated 385 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:21:45.439127 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f2cb4-0397-43c5-bf96-67cddb3c201d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:45.450923 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f2cb4-0397-43c5-bf96-67cddb3c201d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:45.515630 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad7693-a459-4fee-92af-8428520f7097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:45.516241 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad7693-a459-4fee-92af-8428520f7097 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:45.516516 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad7693-a459-4fee-92af-8428520f7097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:45.516782 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad7693-a459-4fee-92af-8428520f7097 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:45.517757 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad7693-a459-4fee-92af-8428520f7097 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:45.520050 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad7693-a459-4fee-92af-8428520f7097 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:45.531494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:45.663084 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1236/2451] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:21:45 2022] POST /identity/v3/users => generated 299 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:21:45.667152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc16329-9b3d-403c-a58c-f3e23379f74e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:45.678023 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc16329-9b3d-403c-a58c-f3e23379f74e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:45.682574 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c0f97-ec31-4842-8ae5-41fc25e34216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:45.683233 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c0f97-ec31-4842-8ae5-41fc25e34216 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:45.689646 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c0f97-ec31-4842-8ae5-41fc25e34216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:45.689646 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c0f97-ec31-4842-8ae5-41fc25e34216 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:45.689646 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c0f97-ec31-4842-8ae5-41fc25e34216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:45.689646 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c0f97-ec31-4842-8ae5-41fc25e34216 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:45.696702 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1216/2452] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:21:45.700985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad7693-a459-4fee-92af-8428520f7097 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:45.711784 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad7693-a459-4fee-92af-8428520f7097 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:45.715984 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c644dd6-2732-435a-813c-de0df5064f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:45.716640 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c644dd6-2732-435a-813c-de0df5064f0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:45.716941 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c644dd6-2732-435a-813c-de0df5064f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:45.717336 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c644dd6-2732-435a-813c-de0df5064f0a admin admin] PATH_INFO: `/v3/projects/e7bc212e7ef44067b561cef134421939/users/6982f5def3624891b6bf96faf0c97530/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:45.733109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c644dd6-2732-435a-813c-de0df5064f0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7bc212e7ef44067b561cef134421939, user_id=6982f5def3624891b6bf96faf0c97530, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:45.735821 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c644dd6-2732-435a-813c-de0df5064f0a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:45.839598 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1237/2453] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:21:45 2022] PUT /identity/v3/projects/e7bc212e7ef44067b561cef134421939/users/6982f5def3624891b6bf96faf0c97530/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:45.843797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c0f97-ec31-4842-8ae5-41fc25e34216 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:45.856735 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c0f97-ec31-4842-8ae5-41fc25e34216 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:45.940267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29cd5c-0b82-42e0-a56e-2603ec35c7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:45.941685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29cd5c-0b82-42e0-a56e-2603ec35c7d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:45.942190 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29cd5c-0b82-42e0-a56e-2603ec35c7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:45.942595 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29cd5c-0b82-42e0-a56e-2603ec35c7d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:45.943706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a29cd5c-0b82-42e0-a56e-2603ec35c7d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:45.945570 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a29cd5c-0b82-42e0-a56e-2603ec35c7d7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:45.965874 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c644dd6-2732-435a-813c-de0df5064f0a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:45.973042 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1217/2454] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:21:45.977056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c644dd6-2732-435a-813c-de0df5064f0a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:45.982577 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8350e2-fc88-4c8b-8b6d-f3c73cba7c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:45.983448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8350e2-fc88-4c8b-8b6d-f3c73cba7c17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:45.983834 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8350e2-fc88-4c8b-8b6d-f3c73cba7c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:45.984196 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8350e2-fc88-4c8b-8b6d-f3c73cba7c17 admin admin] PATH_INFO: `/v3/projects/e7bc212e7ef44067b561cef134421939/users/6982f5def3624891b6bf96faf0c97530/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:45.986428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8350e2-fc88-4c8b-8b6d-f3c73cba7c17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7bc212e7ef44067b561cef134421939, user_id=6982f5def3624891b6bf96faf0c97530, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:45.989346 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8350e2-fc88-4c8b-8b6d-f3c73cba7c17 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:46.002210 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1238/2455] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:21:45 2022] PUT /identity/v3/projects/e7bc212e7ef44067b561cef134421939/users/6982f5def3624891b6bf96faf0c97530/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:46.013119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69ba40-93e2-4e3d-b09c-2becefcefcb3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:46.013400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69ba40-93e2-4e3d-b09c-2becefcefcb3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:46.013670 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69ba40-93e2-4e3d-b09c-2becefcefcb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:46.052289 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3e69ba40-93e2-4e3d-b09c-2becefcefcb3 None None] MFA Rules not processed for user `6982f5def3624891b6bf96faf0c97530`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:21:46.053546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3e69ba40-93e2-4e3d-b09c-2becefcefcb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:46.140444 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1218/2456] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:21:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:21:46.149147 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8350e2-fc88-4c8b-8b6d-f3c73cba7c17 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:46.151024 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4f8350e2-fc88-4c8b-8b6d-f3c73cba7c17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:46.223917 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8350e2-fc88-4c8b-8b6d-f3c73cba7c17 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:46.223917 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b7047-b523-4adb-bfca-fe3dc7300671 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:46.223917 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b7047-b523-4adb-bfca-fe3dc7300671 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:46.223917 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b7047-b523-4adb-bfca-fe3dc7300671 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:46.224463 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b7047-b523-4adb-bfca-fe3dc7300671 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:46.230077 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831b7047-b523-4adb-bfca-fe3dc7300671 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:46.231686 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831b7047-b523-4adb-bfca-fe3dc7300671 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:46.241839 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1239/2457] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:21:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:21:47.661352 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69ba40-93e2-4e3d-b09c-2becefcefcb3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:47.679739 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:47.728982 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69ba40-93e2-4e3d-b09c-2becefcefcb3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:47.756578 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f2b40-0a7f-4354-bff2-ebfe3d3749b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:47.756578 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f2b40-0a7f-4354-bff2-ebfe3d3749b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:47.756578 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f2b40-0a7f-4354-bff2-ebfe3d3749b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:47.756578 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f2b40-0a7f-4354-bff2-ebfe3d3749b4 admin admin] PATH_INFO: `/v3/users/c76daa5ae57646999f4d832a4014a39d` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:47.756943 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f2b40-0a7f-4354-bff2-ebfe3d3749b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c76daa5ae57646999f4d832a4014a39d)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:47.756943 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f2b40-0a7f-4354-bff2-ebfe3d3749b4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:48.035298 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-713f2b40-0a7f-4354-bff2-ebfe3d3749b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c76daa5ae57646999f4d832a4014a39d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:48.156685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-713f2b40-0a7f-4354-bff2-ebfe3d3749b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c76daa5ae57646999f4d832a4014a39d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:48.166955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-713f2b40-0a7f-4354-bff2-ebfe3d3749b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c76daa5ae57646999f4d832a4014a39d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:48.167692 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-713f2b40-0a7f-4354-bff2-ebfe3d3749b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c76daa5ae57646999f4d832a4014a39d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:48.168292 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-713f2b40-0a7f-4354-bff2-ebfe3d3749b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c76daa5ae57646999f4d832a4014a39d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:48.180390 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1219/2458] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:21:47 2022] DELETE /identity/v3/users/c76daa5ae57646999f4d832a4014a39d => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:48.184969 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996962b-2721-4946-a53a-0cea105f6fda None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:48.185425 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996962b-2721-4946-a53a-0cea105f6fda None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:48.185579 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996962b-2721-4946-a53a-0cea105f6fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:48.205227 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d996962b-2721-4946-a53a-0cea105f6fda None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:21:48.207213 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d996962b-2721-4946-a53a-0cea105f6fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:48.291262 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1240/2459] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:21:48.302400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f2b40-0a7f-4354-bff2-ebfe3d3749b4 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:48.304513 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-713f2b40-0a7f-4354-bff2-ebfe3d3749b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:48.386336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f2b40-0a7f-4354-bff2-ebfe3d3749b4 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:48.390916 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5ed7a-2beb-4c92-934a-4e62430f913e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:48.391900 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5ed7a-2beb-4c92-934a-4e62430f913e service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:48.392428 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5ed7a-2beb-4c92-934a-4e62430f913e service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:48.397658 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5ed7a-2beb-4c92-934a-4e62430f913e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:48.403659 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5ed7a-2beb-4c92-934a-4e62430f913e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:48.405494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5ed7a-2beb-4c92-934a-4e62430f913e service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:48.424703 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1220/2460] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:21:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:21:49.139102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996962b-2721-4946-a53a-0cea105f6fda None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:49.140860 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d996962b-2721-4946-a53a-0cea105f6fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:49.154088 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996962b-2721-4946-a53a-0cea105f6fda None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:49.159215 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114dbe0-a41d-4d25-b87d-5e8e97b5de12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:49.160538 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114dbe0-a41d-4d25-b87d-5e8e97b5de12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:49.160885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114dbe0-a41d-4d25-b87d-5e8e97b5de12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:49.161248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114dbe0-a41d-4d25-b87d-5e8e97b5de12 admin admin] PATH_INFO: `/v3/projects/8da1a18135cf4dcba39c05532dbd6a3a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:49.162518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1114dbe0-a41d-4d25-b87d-5e8e97b5de12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8da1a18135cf4dcba39c05532dbd6a3a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:49.164626 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1114dbe0-a41d-4d25-b87d-5e8e97b5de12 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:49.293281 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1114dbe0-a41d-4d25-b87d-5e8e97b5de12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8da1a18135cf4dcba39c05532dbd6a3a', 'request_id': 'req-1114dbe0-a41d-4d25-b87d-5e8e97b5de12'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:49.300396 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1114dbe0-a41d-4d25-b87d-5e8e97b5de12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8da1a18135cf4dcba39c05532dbd6a3a', 'request_id': 'req-1114dbe0-a41d-4d25-b87d-5e8e97b5de12'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:49.303261 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:49.444559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1114dbe0-a41d-4d25-b87d-5e8e97b5de12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8da1a18135cf4dcba39c05532dbd6a3a', 'request_id': 'req-1114dbe0-a41d-4d25-b87d-5e8e97b5de12'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:49.456142 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1114dbe0-a41d-4d25-b87d-5e8e97b5de12 admin admin] The token cache is being invalidate because project 8da1a18135cf4dcba39c05532dbd6a3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:21:49.457223 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1114dbe0-a41d-4d25-b87d-5e8e97b5de12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:49.458342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1114dbe0-a41d-4d25-b87d-5e8e97b5de12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:49.459913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1241/2461] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:21:49 2022] DELETE /identity/v3/projects/8da1a18135cf4dcba39c05532dbd6a3a => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:49.464122 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5ed7a-2beb-4c92-934a-4e62430f913e service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:49.474164 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5ed7a-2beb-4c92-934a-4e62430f913e service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:49.514888 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:49.540625 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a37e3-8299-47b7-8df6-ca7e52e3b4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:49.541280 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a37e3-8299-47b7-8df6-ca7e52e3b4ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:49.541558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a37e3-8299-47b7-8df6-ca7e52e3b4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:49.541859 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a37e3-8299-47b7-8df6-ca7e52e3b4ef admin admin] PATH_INFO: `/v3/users/6982f5def3624891b6bf96faf0c97530` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:49.542969 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a37e3-8299-47b7-8df6-ca7e52e3b4ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=6982f5def3624891b6bf96faf0c97530)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:49.544904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a37e3-8299-47b7-8df6-ca7e52e3b4ef admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:49.887069 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d77a37e3-8299-47b7-8df6-ca7e52e3b4ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6982f5def3624891b6bf96faf0c97530'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:49.970288 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d77a37e3-8299-47b7-8df6-ca7e52e3b4ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6982f5def3624891b6bf96faf0c97530'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:49.983835 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d77a37e3-8299-47b7-8df6-ca7e52e3b4ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6982f5def3624891b6bf96faf0c97530'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:49.984531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d77a37e3-8299-47b7-8df6-ca7e52e3b4ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6982f5def3624891b6bf96faf0c97530'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:49.985139 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d77a37e3-8299-47b7-8df6-ca7e52e3b4ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6982f5def3624891b6bf96faf0c97530'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:50.001762 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1221/2462] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:21:49 2022] DELETE /identity/v3/users/6982f5def3624891b6bf96faf0c97530 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:50.526917 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114dbe0-a41d-4d25-b87d-5e8e97b5de12 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:50.528697 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1114dbe0-a41d-4d25-b87d-5e8e97b5de12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:50.543888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114dbe0-a41d-4d25-b87d-5e8e97b5de12 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:50.622415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10238fb-9373-4652-a834-d19dfb34b26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:50.623069 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10238fb-9373-4652-a834-d19dfb34b26f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:50.623351 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10238fb-9373-4652-a834-d19dfb34b26f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:50.623629 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10238fb-9373-4652-a834-d19dfb34b26f admin admin] PATH_INFO: `/v3/projects/e7bc212e7ef44067b561cef134421939` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:50.624759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10238fb-9373-4652-a834-d19dfb34b26f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7bc212e7ef44067b561cef134421939)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:50.629624 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10238fb-9373-4652-a834-d19dfb34b26f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:50.827142 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f10238fb-9373-4652-a834-d19dfb34b26f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7bc212e7ef44067b561cef134421939', 'request_id': 'req-f10238fb-9373-4652-a834-d19dfb34b26f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:50.836844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f10238fb-9373-4652-a834-d19dfb34b26f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7bc212e7ef44067b561cef134421939', 'request_id': 'req-f10238fb-9373-4652-a834-d19dfb34b26f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:50.953270 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f10238fb-9373-4652-a834-d19dfb34b26f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7bc212e7ef44067b561cef134421939', 'request_id': 'req-f10238fb-9373-4652-a834-d19dfb34b26f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:50.965930 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f10238fb-9373-4652-a834-d19dfb34b26f admin admin] The token cache is being invalidate because project e7bc212e7ef44067b561cef134421939 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:21:50.966568 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f10238fb-9373-4652-a834-d19dfb34b26f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:50.967233 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f10238fb-9373-4652-a834-d19dfb34b26f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:50.973486 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1242/2463] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:21:50 2022] DELETE /identity/v3/projects/e7bc212e7ef44067b561cef134421939 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:50.975536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab3e0b-c201-4b07-81e2-3530b0cab57c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:50.975992 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab3e0b-c201-4b07-81e2-3530b0cab57c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:50.976619 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab3e0b-c201-4b07-81e2-3530b0cab57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:50.998928 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f7ab3e0b-c201-4b07-81e2-3530b0cab57c None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:21:51.000166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f7ab3e0b-c201-4b07-81e2-3530b0cab57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:51.062227 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1222/2464] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:21:51.071192 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbebd3-9a9a-4684-8522-88dea6ded4a4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:51.071522 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbebd3-9a9a-4684-8522-88dea6ded4a4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:51.072008 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbebd3-9a9a-4684-8522-88dea6ded4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:51.091709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-69dbebd3-9a9a-4684-8522-88dea6ded4a4 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:21:51.092928 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-69dbebd3-9a9a-4684-8522-88dea6ded4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:51.101521 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1243/2465] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:21:51.105785 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab3e0b-c201-4b07-81e2-3530b0cab57c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:51.116405 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab3e0b-c201-4b07-81e2-3530b0cab57c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:51.121265 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955a5b3-e1e8-41e6-9339-eb8e3d1c02b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:51.121865 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955a5b3-e1e8-41e6-9339-eb8e3d1c02b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:51.122140 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955a5b3-e1e8-41e6-9339-eb8e3d1c02b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:51.122442 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955a5b3-e1e8-41e6-9339-eb8e3d1c02b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:51.123203 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955a5b3-e1e8-41e6-9339-eb8e3d1c02b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:21:51.123428 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955a5b3-e1e8-41e6-9339-eb8e3d1c02b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:51.124976 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955a5b3-e1e8-41e6-9339-eb8e3d1c02b2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:51.144709 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1223/2466] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:21:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:21:51.145899 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbebd3-9a9a-4684-8522-88dea6ded4a4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:51.157697 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbebd3-9a9a-4684-8522-88dea6ded4a4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:51.163512 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749dc2e1-7838-46c1-b8a4-6b4c4425fd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:51.165550 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749dc2e1-7838-46c1-b8a4-6b4c4425fd87 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:51.165550 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749dc2e1-7838-46c1-b8a4-6b4c4425fd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:51.165550 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749dc2e1-7838-46c1-b8a4-6b4c4425fd87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:51.165550 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749dc2e1-7838-46c1-b8a4-6b4c4425fd87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:51.167229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749dc2e1-7838-46c1-b8a4-6b4c4425fd87 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:51.285668 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1244/2467] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:21:51 2022] POST /identity/v3/projects => generated 381 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:21:51.286658 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955a5b3-e1e8-41e6-9339-eb8e3d1c02b2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:51.299382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955a5b3-e1e8-41e6-9339-eb8e3d1c02b2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:51.381354 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:51.403745 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cb270-94a3-4440-bd36-89eb199afdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:51.404493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cb270-94a3-4440-bd36-89eb199afdb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:51.404976 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cb270-94a3-4440-bd36-89eb199afdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:51.405543 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cb270-94a3-4440-bd36-89eb199afdb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:51.406264 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79cb270-94a3-4440-bd36-89eb199afdb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:51.416967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79cb270-94a3-4440-bd36-89eb199afdb9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:51.492471 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749dc2e1-7838-46c1-b8a4-6b4c4425fd87 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:51.496841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-749dc2e1-7838-46c1-b8a4-6b4c4425fd87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:51.520828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1224/2468] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:21:51 2022] POST /identity/v3/users => generated 291 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:21:51.524989 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cb270-94a3-4440-bd36-89eb199afdb9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:51.536413 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cb270-94a3-4440-bd36-89eb199afdb9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:51.541736 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ccdff2-db29-4623-b6ee-21daa9ea9bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:51.542503 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccdff2-db29-4623-b6ee-21daa9ea9bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:51.542920 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccdff2-db29-4623-b6ee-21daa9ea9bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:51.543358 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccdff2-db29-4623-b6ee-21daa9ea9bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:51.544055 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ccdff2-db29-4623-b6ee-21daa9ea9bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:51.546424 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ccdff2-db29-4623-b6ee-21daa9ea9bd4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:51.558584 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1225/2469] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:21:51.562318 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ccdff2-db29-4623-b6ee-21daa9ea9bd4 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:51.568988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:51.581444 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749dc2e1-7838-46c1-b8a4-6b4c4425fd87 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:51.581668 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ccdff2-db29-4623-b6ee-21daa9ea9bd4 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:51.581781 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c71bd8a-f152-4cb2-b8cf-ddf0ca9bbdc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:51.581781 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71bd8a-f152-4cb2-b8cf-ddf0ca9bbdc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:51.581781 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71bd8a-f152-4cb2-b8cf-ddf0ca9bbdc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:51.581781 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71bd8a-f152-4cb2-b8cf-ddf0ca9bbdc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:51.582110 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8c71bd8a-f152-4cb2-b8cf-ddf0ca9bbdc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:51.582187 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e99f4-1c81-4747-82e8-90d8ac16eb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:51.582663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e99f4-1c81-4747-82e8-90d8ac16eb99 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:51.582935 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e99f4-1c81-4747-82e8-90d8ac16eb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:51.583435 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e99f4-1c81-4747-82e8-90d8ac16eb99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:51.584089 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e99f4-1c81-4747-82e8-90d8ac16eb99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:51.585882 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e99f4-1c81-4747-82e8-90d8ac16eb99 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:51.594584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c71bd8a-f152-4cb2-b8cf-ddf0ca9bbdc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:51.596110 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c71bd8a-f152-4cb2-b8cf-ddf0ca9bbdc5 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:51.598675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1226/2470] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:21:51.602418 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e99f4-1c81-4747-82e8-90d8ac16eb99 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:51.616965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e99f4-1c81-4747-82e8-90d8ac16eb99 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:51.622283 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1245/2471] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:21:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:21:51.622620 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c33ef-575f-44ef-be14-b6c9811722a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:51.623315 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c33ef-575f-44ef-be14-b6c9811722a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:51.623609 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c33ef-575f-44ef-be14-b6c9811722a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:51.623882 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c33ef-575f-44ef-be14-b6c9811722a5 admin admin] PATH_INFO: `/v3/projects/8febb4462ee040c1956f29d1a1728e17/users/e8dddd389eab4219a422d79a8b574b71/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:51.644475 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13c33ef-575f-44ef-be14-b6c9811722a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8febb4462ee040c1956f29d1a1728e17, user_id=e8dddd389eab4219a422d79a8b574b71, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:51.648636 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13c33ef-575f-44ef-be14-b6c9811722a5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:51.737184 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1227/2472] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:21:51 2022] PUT /identity/v3/projects/8febb4462ee040c1956f29d1a1728e17/users/e8dddd389eab4219a422d79a8b574b71/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:21:51.748474 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17505cc-7797-4c7c-8ca2-977ee352968d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:51.748799 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17505cc-7797-4c7c-8ca2-977ee352968d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:51.749107 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17505cc-7797-4c7c-8ca2-977ee352968d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:51.778924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a17505cc-7797-4c7c-8ca2-977ee352968d None None] MFA Rules not processed for user `e8dddd389eab4219a422d79a8b574b71`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:21:51.780112 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a17505cc-7797-4c7c-8ca2-977ee352968d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:51.887611 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1246/2473] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:21:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:21:51.904203 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c33ef-575f-44ef-be14-b6c9811722a5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:51.910760 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a13c33ef-575f-44ef-be14-b6c9811722a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:51.999643 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c33ef-575f-44ef-be14-b6c9811722a5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:52.004418 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cb572-96f4-40c7-a810-c5a37510169c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:52.005041 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cb572-96f4-40c7-a810-c5a37510169c service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:52.005314 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cb572-96f4-40c7-a810-c5a37510169c service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:52.006182 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cb572-96f4-40c7-a810-c5a37510169c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:52.020122 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622cb572-96f4-40c7-a810-c5a37510169c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:52.021657 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622cb572-96f4-40c7-a810-c5a37510169c service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:52.033101 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1228/2474] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:21:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:21:53.351069 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17505cc-7797-4c7c-8ca2-977ee352968d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:53.352672 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a17505cc-7797-4c7c-8ca2-977ee352968d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:53.434654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17505cc-7797-4c7c-8ca2-977ee352968d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:53.440172 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f1549-ed16-4615-a4b8-04e6b05d5721 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cd39d9a583a4d73a5654017be106019', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:53.440851 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f1549-ed16-4615-a4b8-04e6b05d5721 service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:53.441225 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f1549-ed16-4615-a4b8-04e6b05d5721 service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:53.441225 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f1549-ed16-4615-a4b8-04e6b05d5721 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:53.443334 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3d8f1549-ed16-4615-a4b8-04e6b05d5721 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:53.456270 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f1549-ed16-4615-a4b8-04e6b05d5721 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:53.459457 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f1549-ed16-4615-a4b8-04e6b05d5721 service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:53.483241 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cb572-96f4-40c7-a810-c5a37510169c service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:53.486281 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1247/2475] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:21:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:21:53.486676 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-622cb572-96f4-40c7-a810-c5a37510169c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:53.574827 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cb572-96f4-40c7-a810-c5a37510169c service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:53.579713 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38eece-e051-4d2f-be27-db39cb69482e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:53.580559 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38eece-e051-4d2f-be27-db39cb69482e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:53.580559 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38eece-e051-4d2f-be27-db39cb69482e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:53.580744 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38eece-e051-4d2f-be27-db39cb69482e admin admin] PATH_INFO: `/v3/users/22a5224313e04d35af7fc7802ef5d6ed` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:53.582389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b38eece-e051-4d2f-be27-db39cb69482e admin admin] RBAC: Authorizing `identity:delete_user(user_id=22a5224313e04d35af7fc7802ef5d6ed)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:53.584875 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b38eece-e051-4d2f-be27-db39cb69482e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:53.742752 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:53.761415 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8b38eece-e051-4d2f-be27-db39cb69482e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22a5224313e04d35af7fc7802ef5d6ed'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:53.855981 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8b38eece-e051-4d2f-be27-db39cb69482e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22a5224313e04d35af7fc7802ef5d6ed'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:53.873893 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8b38eece-e051-4d2f-be27-db39cb69482e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22a5224313e04d35af7fc7802ef5d6ed'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:53.875196 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8b38eece-e051-4d2f-be27-db39cb69482e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22a5224313e04d35af7fc7802ef5d6ed'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:53.876192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8b38eece-e051-4d2f-be27-db39cb69482e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22a5224313e04d35af7fc7802ef5d6ed'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:53.898379 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1229/2476] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:21:53 2022] DELETE /identity/v3/users/22a5224313e04d35af7fc7802ef5d6ed => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:54.258964 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f1549-ed16-4615-a4b8-04e6b05d5721 service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:54.279660 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f1549-ed16-4615-a4b8-04e6b05d5721 service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:54.348757 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d8ec3-b662-41b9-a0d7-54cf650bf60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:54.349430 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d8ec3-b662-41b9-a0d7-54cf650bf60c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:54.349742 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d8ec3-b662-41b9-a0d7-54cf650bf60c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:54.350149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d8ec3-b662-41b9-a0d7-54cf650bf60c admin admin] PATH_INFO: `/v3/projects/e1bc63782f1d4b32ab58852232df16ac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:54.351435 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d8ec3-b662-41b9-a0d7-54cf650bf60c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1bc63782f1d4b32ab58852232df16ac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:54.353397 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d8ec3-b662-41b9-a0d7-54cf650bf60c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:54.367298 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:54.493703 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-775d8ec3-b662-41b9-a0d7-54cf650bf60c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1bc63782f1d4b32ab58852232df16ac', 'request_id': 'req-775d8ec3-b662-41b9-a0d7-54cf650bf60c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:54.505351 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-775d8ec3-b662-41b9-a0d7-54cf650bf60c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1bc63782f1d4b32ab58852232df16ac', 'request_id': 'req-775d8ec3-b662-41b9-a0d7-54cf650bf60c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:54.569963 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-775d8ec3-b662-41b9-a0d7-54cf650bf60c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1bc63782f1d4b32ab58852232df16ac', 'request_id': 'req-775d8ec3-b662-41b9-a0d7-54cf650bf60c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:54.582494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-775d8ec3-b662-41b9-a0d7-54cf650bf60c admin admin] The token cache is being invalidate because project e1bc63782f1d4b32ab58852232df16ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:21:54.582783 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-775d8ec3-b662-41b9-a0d7-54cf650bf60c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:54.583477 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-775d8ec3-b662-41b9-a0d7-54cf650bf60c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:54.585024 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1248/2477] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:21:54 2022] DELETE /identity/v3/projects/e1bc63782f1d4b32ab58852232df16ac => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:54.589102 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38eece-e051-4d2f-be27-db39cb69482e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:54.590631 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8b38eece-e051-4d2f-be27-db39cb69482e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:54.608211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38eece-e051-4d2f-be27-db39cb69482e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:54.665834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0e8d0-f9ea-4008-859a-d710ad9e2e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:54.666498 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0e8d0-f9ea-4008-859a-d710ad9e2e5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:54.666774 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0e8d0-f9ea-4008-859a-d710ad9e2e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:54.667047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0e8d0-f9ea-4008-859a-d710ad9e2e5f admin admin] PATH_INFO: `/v3/users/b21eaa7c53f64f2a94e29f9b24efaef9` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:54.668191 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0e8d0-f9ea-4008-859a-d710ad9e2e5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b21eaa7c53f64f2a94e29f9b24efaef9)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:54.670097 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0e8d0-f9ea-4008-859a-d710ad9e2e5f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:54.972449 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b7b0e8d0-f9ea-4008-859a-d710ad9e2e5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b21eaa7c53f64f2a94e29f9b24efaef9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:55.090324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b7b0e8d0-f9ea-4008-859a-d710ad9e2e5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b21eaa7c53f64f2a94e29f9b24efaef9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:55.111216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b7b0e8d0-f9ea-4008-859a-d710ad9e2e5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b21eaa7c53f64f2a94e29f9b24efaef9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:55.111853 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b7b0e8d0-f9ea-4008-859a-d710ad9e2e5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b21eaa7c53f64f2a94e29f9b24efaef9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:55.112467 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b7b0e8d0-f9ea-4008-859a-d710ad9e2e5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b21eaa7c53f64f2a94e29f9b24efaef9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:55.144844 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1230/2478] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:21:54 2022] DELETE /identity/v3/users/b21eaa7c53f64f2a94e29f9b24efaef9 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:55.486980 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d8ec3-b662-41b9-a0d7-54cf650bf60c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:55.492604 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-775d8ec3-b662-41b9-a0d7-54cf650bf60c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:55.513391 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d8ec3-b662-41b9-a0d7-54cf650bf60c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:55.583939 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeee1c6-ddfe-4a2d-b00d-c0f39b529d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:55.584686 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeee1c6-ddfe-4a2d-b00d-c0f39b529d5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:55.585169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeee1c6-ddfe-4a2d-b00d-c0f39b529d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:55.585600 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeee1c6-ddfe-4a2d-b00d-c0f39b529d5f admin admin] PATH_INFO: `/v3/projects/ef4319031106499c8cd532dfce92cc89` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:55.586898 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeee1c6-ddfe-4a2d-b00d-c0f39b529d5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef4319031106499c8cd532dfce92cc89)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:55.589269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeee1c6-ddfe-4a2d-b00d-c0f39b529d5f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:55.697311 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-caeee1c6-ddfe-4a2d-b00d-c0f39b529d5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef4319031106499c8cd532dfce92cc89', 'request_id': 'req-caeee1c6-ddfe-4a2d-b00d-c0f39b529d5f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:55.704440 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-caeee1c6-ddfe-4a2d-b00d-c0f39b529d5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef4319031106499c8cd532dfce92cc89', 'request_id': 'req-caeee1c6-ddfe-4a2d-b00d-c0f39b529d5f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:55.776287 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-caeee1c6-ddfe-4a2d-b00d-c0f39b529d5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef4319031106499c8cd532dfce92cc89', 'request_id': 'req-caeee1c6-ddfe-4a2d-b00d-c0f39b529d5f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:55.790262 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-caeee1c6-ddfe-4a2d-b00d-c0f39b529d5f admin admin] The token cache is being invalidate because project ef4319031106499c8cd532dfce92cc89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:21:55.790590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-caeee1c6-ddfe-4a2d-b00d-c0f39b529d5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:55.791746 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-caeee1c6-ddfe-4a2d-b00d-c0f39b529d5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:55.793247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1249/2479] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:21:55 2022] DELETE /identity/v3/projects/ef4319031106499c8cd532dfce92cc89 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:55.950751 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0e8d0-f9ea-4008-859a-d710ad9e2e5f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:55.953259 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b7b0e8d0-f9ea-4008-859a-d710ad9e2e5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:55.970446 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0e8d0-f9ea-4008-859a-d710ad9e2e5f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:55.998798 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:56.031313 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746882a5-2974-4698-b777-84bdc1564609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:56.032013 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746882a5-2974-4698-b777-84bdc1564609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:56.032294 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746882a5-2974-4698-b777-84bdc1564609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:56.032559 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746882a5-2974-4698-b777-84bdc1564609 admin admin] PATH_INFO: `/v3/users/48feb2fc02f04e739c5e1bf7d68baa16` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:56.033698 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746882a5-2974-4698-b777-84bdc1564609 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48feb2fc02f04e739c5e1bf7d68baa16)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:56.036217 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746882a5-2974-4698-b777-84bdc1564609 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:56.326940 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-746882a5-2974-4698-b777-84bdc1564609 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48feb2fc02f04e739c5e1bf7d68baa16'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:56.398713 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-746882a5-2974-4698-b777-84bdc1564609 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48feb2fc02f04e739c5e1bf7d68baa16'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:56.412387 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-746882a5-2974-4698-b777-84bdc1564609 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48feb2fc02f04e739c5e1bf7d68baa16'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:56.413053 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-746882a5-2974-4698-b777-84bdc1564609 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48feb2fc02f04e739c5e1bf7d68baa16'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:56.413651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-746882a5-2974-4698-b777-84bdc1564609 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48feb2fc02f04e739c5e1bf7d68baa16'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:56.426687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1231/2480] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:21:55 2022] DELETE /identity/v3/users/48feb2fc02f04e739c5e1bf7d68baa16 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:56.685698 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeee1c6-ddfe-4a2d-b00d-c0f39b529d5f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:56.687261 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-caeee1c6-ddfe-4a2d-b00d-c0f39b529d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:56.704198 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeee1c6-ddfe-4a2d-b00d-c0f39b529d5f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:56.729247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:56.759046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1f6e8-ff7b-4000-836f-04fe47742b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:56.759980 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1f6e8-ff7b-4000-836f-04fe47742b42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:56.760680 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1f6e8-ff7b-4000-836f-04fe47742b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:56.761103 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1f6e8-ff7b-4000-836f-04fe47742b42 admin admin] PATH_INFO: `/v3/projects/cfac2c11f1b24456a5a9efaa9112e281` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:56.762301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1f6e8-ff7b-4000-836f-04fe47742b42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfac2c11f1b24456a5a9efaa9112e281)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:56.764245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1f6e8-ff7b-4000-836f-04fe47742b42 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:56.879343 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-21e1f6e8-ff7b-4000-836f-04fe47742b42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfac2c11f1b24456a5a9efaa9112e281', 'request_id': 'req-21e1f6e8-ff7b-4000-836f-04fe47742b42'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:56.885886 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-21e1f6e8-ff7b-4000-836f-04fe47742b42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfac2c11f1b24456a5a9efaa9112e281', 'request_id': 'req-21e1f6e8-ff7b-4000-836f-04fe47742b42'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:56.969729 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-21e1f6e8-ff7b-4000-836f-04fe47742b42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfac2c11f1b24456a5a9efaa9112e281', 'request_id': 'req-21e1f6e8-ff7b-4000-836f-04fe47742b42'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:56.982002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-21e1f6e8-ff7b-4000-836f-04fe47742b42 admin admin] The token cache is being invalidate because project cfac2c11f1b24456a5a9efaa9112e281 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:21:56.982281 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-21e1f6e8-ff7b-4000-836f-04fe47742b42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:56.982936 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-21e1f6e8-ff7b-4000-836f-04fe47742b42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:21:56.984417 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1250/2481] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:21:56 2022] DELETE /identity/v3/projects/cfac2c11f1b24456a5a9efaa9112e281 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:56.989881 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a97d2f-0e16-4366-b446-846fd5db144b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:56.990383 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a97d2f-0e16-4366-b446-846fd5db144b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:56.990483 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a97d2f-0e16-4366-b446-846fd5db144b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:57.011048 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-38a97d2f-0e16-4366-b446-846fd5db144b None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:21:57.012237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-38a97d2f-0e16-4366-b446-846fd5db144b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:57.071565 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1232/2482] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:21:57.079494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ea6aa-f348-4bfe-aefa-7f5447686e72 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:57.079856 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ea6aa-f348-4bfe-aefa-7f5447686e72 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:57.080220 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ea6aa-f348-4bfe-aefa-7f5447686e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:57.100402 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e72ea6aa-f348-4bfe-aefa-7f5447686e72 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:21:57.101602 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e72ea6aa-f348-4bfe-aefa-7f5447686e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:57.109726 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1251/2483] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:21:57.122803 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a97d2f-0e16-4366-b446-846fd5db144b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:57.133892 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a97d2f-0e16-4366-b446-846fd5db144b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:57.138542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b616144-2b68-4f16-aefb-1de1d02f7a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:57.139136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b616144-2b68-4f16-aefb-1de1d02f7a14 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:57.139415 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b616144-2b68-4f16-aefb-1de1d02f7a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:57.139743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b616144-2b68-4f16-aefb-1de1d02f7a14 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:57.140395 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b616144-2b68-4f16-aefb-1de1d02f7a14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:21:57.140703 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b616144-2b68-4f16-aefb-1de1d02f7a14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:57.142289 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b616144-2b68-4f16-aefb-1de1d02f7a14 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:57.155169 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1233/2484] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:21:57.159514 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ea6aa-f348-4bfe-aefa-7f5447686e72 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:57.168821 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ea6aa-f348-4bfe-aefa-7f5447686e72 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:57.173556 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee4df7-dad2-4a2d-821f-19131d689b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:57.174117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee4df7-dad2-4a2d-821f-19131d689b63 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:57.174430 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee4df7-dad2-4a2d-821f-19131d689b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:57.174710 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee4df7-dad2-4a2d-821f-19131d689b63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:57.175332 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ee4df7-dad2-4a2d-821f-19131d689b63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:57.177237 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ee4df7-dad2-4a2d-821f-19131d689b63 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:57.297706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1252/2485] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:21:57 2022] POST /identity/v3/projects => generated 369 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:21:57.302514 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b616144-2b68-4f16-aefb-1de1d02f7a14 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:57.312592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b616144-2b68-4f16-aefb-1de1d02f7a14 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:57.344442 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:57.371461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4b2b3-23ce-4473-9011-23b5427fee00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:57.372056 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4b2b3-23ce-4473-9011-23b5427fee00 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:57.372330 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4b2b3-23ce-4473-9011-23b5427fee00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:57.372597 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4b2b3-23ce-4473-9011-23b5427fee00 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:57.373219 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4b2b3-23ce-4473-9011-23b5427fee00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:57.375190 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4b2b3-23ce-4473-9011-23b5427fee00 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:57.455822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1234/2486] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:21:57 2022] POST /identity/v3/users => generated 285 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:21:57.460668 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee4df7-dad2-4a2d-821f-19131d689b63 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:57.470952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee4df7-dad2-4a2d-821f-19131d689b63 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:57.476481 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b9b01-1f6c-41e8-bfe6-4dc0b539ee62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:57.477096 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b9b01-1f6c-41e8-bfe6-4dc0b539ee62 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:57.477372 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b9b01-1f6c-41e8-bfe6-4dc0b539ee62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:57.477640 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b9b01-1f6c-41e8-bfe6-4dc0b539ee62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:57.478281 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b9b01-1f6c-41e8-bfe6-4dc0b539ee62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:57.479903 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b9b01-1f6c-41e8-bfe6-4dc0b539ee62 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:57.490085 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1253/2487] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:21:57.494180 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4b2b3-23ce-4473-9011-23b5427fee00 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:57.504699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4b2b3-23ce-4473-9011-23b5427fee00 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:57.509739 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b501505-0938-4ab0-843d-0b18062c3460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:57.510338 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b501505-0938-4ab0-843d-0b18062c3460 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:57.510648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b501505-0938-4ab0-843d-0b18062c3460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:57.510921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b501505-0938-4ab0-843d-0b18062c3460 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:57.511603 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b501505-0938-4ab0-843d-0b18062c3460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:57.513203 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b501505-0938-4ab0-843d-0b18062c3460 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:57.523921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1235/2488] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:21:57.528971 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b9b01-1f6c-41e8-bfe6-4dc0b539ee62 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:57.539685 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b9b01-1f6c-41e8-bfe6-4dc0b539ee62 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:57.544352 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa66fee-cc2d-47a6-847c-426587c56bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:57.545017 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa66fee-cc2d-47a6-847c-426587c56bf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:57.545294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa66fee-cc2d-47a6-847c-426587c56bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:57.545564 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa66fee-cc2d-47a6-847c-426587c56bf2 admin admin] PATH_INFO: `/v3/projects/242c8af510fe476085e5766b88085283/users/8dded5e9c458496185063177ca60ba1f/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:57.561026 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa66fee-cc2d-47a6-847c-426587c56bf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=242c8af510fe476085e5766b88085283, user_id=8dded5e9c458496185063177ca60ba1f, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:57.563706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa66fee-cc2d-47a6-847c-426587c56bf2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:57.644906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1254/2489] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:21:57 2022] PUT /identity/v3/projects/242c8af510fe476085e5766b88085283/users/8dded5e9c458496185063177ca60ba1f/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:21:57.655962 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ef12b-ed44-46d9-9655-6df0adff0c1f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:57.656249 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ef12b-ed44-46d9-9655-6df0adff0c1f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:57.660621 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ef12b-ed44-46d9-9655-6df0adff0c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:57.685297 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c85ef12b-ed44-46d9-9655-6df0adff0c1f None None] MFA Rules not processed for user `8dded5e9c458496185063177ca60ba1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:21:57.686547 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c85ef12b-ed44-46d9-9655-6df0adff0c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:57.749170 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1236/2490] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:21:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:21:57.754168 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa66fee-cc2d-47a6-847c-426587c56bf2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:57.811717 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:57.816856 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa66fee-cc2d-47a6-847c-426587c56bf2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:57.821750 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e602054-6cbd-4ecd-bd72-e96b41e572ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:57.822366 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e602054-6cbd-4ecd-bd72-e96b41e572ff admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:57.822654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e602054-6cbd-4ecd-bd72-e96b41e572ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:57.824265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e602054-6cbd-4ecd-bd72-e96b41e572ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:57.824265 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e602054-6cbd-4ecd-bd72-e96b41e572ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:57.825480 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e602054-6cbd-4ecd-bd72-e96b41e572ff admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:57.907884 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1255/2491] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:21:57 2022] POST /identity/v3/projects => generated 367 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:21:57.912216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ef12b-ed44-46d9-9655-6df0adff0c1f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:57.922869 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ef12b-ed44-46d9-9655-6df0adff0c1f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:57.989442 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c188054-94b5-448c-a8c3-318346334d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:57.990041 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c188054-94b5-448c-a8c3-318346334d46 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:57.990666 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c188054-94b5-448c-a8c3-318346334d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:57.990666 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c188054-94b5-448c-a8c3-318346334d46 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:57.991224 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c188054-94b5-448c-a8c3-318346334d46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:57.993188 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c188054-94b5-448c-a8c3-318346334d46 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:58.124320 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1237/2492] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:21:57 2022] POST /identity/v3/users => generated 290 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:21:58.129187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e602054-6cbd-4ecd-bd72-e96b41e572ff admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:58.139705 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e602054-6cbd-4ecd-bd72-e96b41e572ff admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:58.144289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc1e5de-7134-4a20-a463-08db122d09a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:58.144920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1e5de-7134-4a20-a463-08db122d09a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:58.145200 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1e5de-7134-4a20-a463-08db122d09a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:58.145474 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1e5de-7134-4a20-a463-08db122d09a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:58.146129 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc1e5de-7134-4a20-a463-08db122d09a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:58.147776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc1e5de-7134-4a20-a463-08db122d09a6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:58.157776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1256/2493] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:21:58.162106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c188054-94b5-448c-a8c3-318346334d46 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:58.171897 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c188054-94b5-448c-a8c3-318346334d46 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:58.177102 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db808cb-af0a-4fac-9a0f-67480eba9f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:58.177742 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db808cb-af0a-4fac-9a0f-67480eba9f07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:58.178011 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db808cb-af0a-4fac-9a0f-67480eba9f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:58.178268 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db808cb-af0a-4fac-9a0f-67480eba9f07 admin admin] PATH_INFO: `/v3/projects/2fbe3198d4034a7b867dc0ed33629d64/users/f7bb80a0d246414c95ac72d55947ac2a/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:58.193428 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db808cb-af0a-4fac-9a0f-67480eba9f07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fbe3198d4034a7b867dc0ed33629d64, user_id=f7bb80a0d246414c95ac72d55947ac2a, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:58.196092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db808cb-af0a-4fac-9a0f-67480eba9f07 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:58.274716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1238/2494] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:21:58 2022] PUT /identity/v3/projects/2fbe3198d4034a7b867dc0ed33629d64/users/f7bb80a0d246414c95ac72d55947ac2a/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:58.278789 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc1e5de-7134-4a20-a463-08db122d09a6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:58.290771 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc1e5de-7134-4a20-a463-08db122d09a6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:58.347621 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd873f-9505-4b40-b29b-62ea24feec03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:58.348246 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd873f-9505-4b40-b29b-62ea24feec03 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:58.348527 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd873f-9505-4b40-b29b-62ea24feec03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:58.348801 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd873f-9505-4b40-b29b-62ea24feec03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:58.349438 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd873f-9505-4b40-b29b-62ea24feec03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:58.351066 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd873f-9505-4b40-b29b-62ea24feec03 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:58.360834 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1257/2495] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:21:58.365235 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db808cb-af0a-4fac-9a0f-67480eba9f07 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:58.375638 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db808cb-af0a-4fac-9a0f-67480eba9f07 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:58.381045 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33aeb6-af62-49b8-9507-534fd024eb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:58.381710 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33aeb6-af62-49b8-9507-534fd024eb28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:58.381986 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33aeb6-af62-49b8-9507-534fd024eb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:58.382256 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33aeb6-af62-49b8-9507-534fd024eb28 admin admin] PATH_INFO: `/v3/projects/2fbe3198d4034a7b867dc0ed33629d64/users/f7bb80a0d246414c95ac72d55947ac2a/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:58.384453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33aeb6-af62-49b8-9507-534fd024eb28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fbe3198d4034a7b867dc0ed33629d64, user_id=f7bb80a0d246414c95ac72d55947ac2a, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:58.387155 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33aeb6-af62-49b8-9507-534fd024eb28 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:58.389344 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:58.398254 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1239/2496] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:21:58 2022] PUT /identity/v3/projects/2fbe3198d4034a7b867dc0ed33629d64/users/f7bb80a0d246414c95ac72d55947ac2a/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:21:58.410793 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6725252-71c6-4bf0-b4e9-fb71d92067f1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:58.411109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6725252-71c6-4bf0-b4e9-fb71d92067f1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:58.411388 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6725252-71c6-4bf0-b4e9-fb71d92067f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:58.438815 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d6725252-71c6-4bf0-b4e9-fb71d92067f1 None None] MFA Rules not processed for user `f7bb80a0d246414c95ac72d55947ac2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:21:58.440025 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d6725252-71c6-4bf0-b4e9-fb71d92067f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:58.514751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1258/2497] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:21:58 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:21:58.526704 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33aeb6-af62-49b8-9507-534fd024eb28 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:58.528945 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cb33aeb6-af62-49b8-9507-534fd024eb28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:21:58.588059 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33aeb6-af62-49b8-9507-534fd024eb28 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:58.591769 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3936910-d17f-4faa-8826-10947190eb60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:58.592361 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3936910-d17f-4faa-8826-10947190eb60 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:58.592692 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3936910-d17f-4faa-8826-10947190eb60 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:58.592969 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3936910-d17f-4faa-8826-10947190eb60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:58.602528 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3936910-d17f-4faa-8826-10947190eb60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:58.603969 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3936910-d17f-4faa-8826-10947190eb60 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:58.614536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1240/2498] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:21:58 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:21:58.762523 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6725252-71c6-4bf0-b4e9-fb71d92067f1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:21:58.773094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6725252-71c6-4bf0-b4e9-fb71d92067f1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:21:58.776777 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24689abe-6926-4881-bb32-7061dc8b15ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:21:58.777375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24689abe-6926-4881-bb32-7061dc8b15ed service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:21:58.777630 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24689abe-6926-4881-bb32-7061dc8b15ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:21:58.777892 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24689abe-6926-4881-bb32-7061dc8b15ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:21:58.824995 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:21:58.835508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24689abe-6926-4881-bb32-7061dc8b15ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:21:58.836958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24689abe-6926-4881-bb32-7061dc8b15ed service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:21:58.845764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1259/2499] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:21:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:22:02.638011 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3936910-d17f-4faa-8826-10947190eb60 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:02.639466 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f3936910-d17f-4faa-8826-10947190eb60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:02.704038 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3936910-d17f-4faa-8826-10947190eb60 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:02.709313 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d05b3-e91c-455b-a678-031a2f58cf13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:02.709938 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d05b3-e91c-455b-a678-031a2f58cf13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:02.710210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d05b3-e91c-455b-a678-031a2f58cf13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:02.710508 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d05b3-e91c-455b-a678-031a2f58cf13 admin admin] PATH_INFO: `/v3/users/e8dddd389eab4219a422d79a8b574b71` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:02.711666 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d05b3-e91c-455b-a678-031a2f58cf13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8dddd389eab4219a422d79a8b574b71)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:02.713580 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d05b3-e91c-455b-a678-031a2f58cf13 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:02.943801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-489d05b3-e91c-455b-a678-031a2f58cf13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8dddd389eab4219a422d79a8b574b71'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:03.010085 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-489d05b3-e91c-455b-a678-031a2f58cf13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8dddd389eab4219a422d79a8b574b71'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:03.024211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-489d05b3-e91c-455b-a678-031a2f58cf13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8dddd389eab4219a422d79a8b574b71'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:03.024908 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-489d05b3-e91c-455b-a678-031a2f58cf13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8dddd389eab4219a422d79a8b574b71'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:03.025540 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-489d05b3-e91c-455b-a678-031a2f58cf13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8dddd389eab4219a422d79a8b574b71'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:03.039228 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1241/2500] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:22:02 2022] DELETE /identity/v3/users/e8dddd389eab4219a422d79a8b574b71 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:03.043719 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63091fa6-fe0e-4eac-8b70-b94faafeb834 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:03.048660 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63091fa6-fe0e-4eac-8b70-b94faafeb834 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:03.048660 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63091fa6-fe0e-4eac-8b70-b94faafeb834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:03.063494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-63091fa6-fe0e-4eac-8b70-b94faafeb834 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:03.064687 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-63091fa6-fe0e-4eac-8b70-b94faafeb834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:03.124999 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1260/2501] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:03.135354 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d05b3-e91c-455b-a678-031a2f58cf13 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:03.136738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-489d05b3-e91c-455b-a678-031a2f58cf13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:03.155293 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:03.194289 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d05b3-e91c-455b-a678-031a2f58cf13 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:03.197897 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6251a-3a59-48bf-9b6e-c7a8f67d785b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:03.198535 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6251a-3a59-48bf-9b6e-c7a8f67d785b service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:03.198814 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6251a-3a59-48bf-9b6e-c7a8f67d785b service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:03.199085 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6251a-3a59-48bf-9b6e-c7a8f67d785b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:03.209148 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6251a-3a59-48bf-9b6e-c7a8f67d785b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:03.210620 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6251a-3a59-48bf-9b6e-c7a8f67d785b service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:03.222684 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1242/2502] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:22:03.646447 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63091fa6-fe0e-4eac-8b70-b94faafeb834 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:03.657501 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-63091fa6-fe0e-4eac-8b70-b94faafeb834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:03.661378 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63091fa6-fe0e-4eac-8b70-b94faafeb834 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:03.666100 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fabe69e-8ad2-4937-a11b-a6b4fc34e980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:03.666756 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabe69e-8ad2-4937-a11b-a6b4fc34e980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:03.667037 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabe69e-8ad2-4937-a11b-a6b4fc34e980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:03.667312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabe69e-8ad2-4937-a11b-a6b4fc34e980 admin admin] PATH_INFO: `/v3/projects/8febb4462ee040c1956f29d1a1728e17` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:03.668431 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fabe69e-8ad2-4937-a11b-a6b4fc34e980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8febb4462ee040c1956f29d1a1728e17)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:03.670343 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fabe69e-8ad2-4937-a11b-a6b4fc34e980 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:03.781849 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5fabe69e-8ad2-4937-a11b-a6b4fc34e980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8febb4462ee040c1956f29d1a1728e17', 'request_id': 'req-5fabe69e-8ad2-4937-a11b-a6b4fc34e980'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:03.788859 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5fabe69e-8ad2-4937-a11b-a6b4fc34e980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8febb4462ee040c1956f29d1a1728e17', 'request_id': 'req-5fabe69e-8ad2-4937-a11b-a6b4fc34e980'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:03.850272 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5fabe69e-8ad2-4937-a11b-a6b4fc34e980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8febb4462ee040c1956f29d1a1728e17', 'request_id': 'req-5fabe69e-8ad2-4937-a11b-a6b4fc34e980'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:03.866096 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5fabe69e-8ad2-4937-a11b-a6b4fc34e980 admin admin] The token cache is being invalidate because project 8febb4462ee040c1956f29d1a1728e17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:22:03.866415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5fabe69e-8ad2-4937-a11b-a6b4fc34e980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:03.867080 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5fabe69e-8ad2-4937-a11b-a6b4fc34e980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:03.868567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1261/2503] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:22:03 2022] DELETE /identity/v3/projects/8febb4462ee040c1956f29d1a1728e17 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:03.873511 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641b05b-9460-4998-a31c-12ed02c7a491 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:03.873996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641b05b-9460-4998-a31c-12ed02c7a491 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:03.874107 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641b05b-9460-4998-a31c-12ed02c7a491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:03.895048 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6641b05b-9460-4998-a31c-12ed02c7a491 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:03.897173 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6641b05b-9460-4998-a31c-12ed02c7a491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:03.958023 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1243/2504] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:03.967508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a48f66-8d46-4cff-a665-34c00eb180ce None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:03.967816 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a48f66-8d46-4cff-a665-34c00eb180ce None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:03.968192 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a48f66-8d46-4cff-a665-34c00eb180ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:03.988005 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-40a48f66-8d46-4cff-a665-34c00eb180ce None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:03.989231 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-40a48f66-8d46-4cff-a665-34c00eb180ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:03.997039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1262/2505] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:04.001316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641b05b-9460-4998-a31c-12ed02c7a491 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:04.011950 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641b05b-9460-4998-a31c-12ed02c7a491 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:04.017775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeeb608-b175-4258-89fa-bf2eff10121d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:04.018452 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeeb608-b175-4258-89fa-bf2eff10121d admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:04.018714 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeeb608-b175-4258-89fa-bf2eff10121d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:04.019020 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeeb608-b175-4258-89fa-bf2eff10121d admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:04.021595 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeeb608-b175-4258-89fa-bf2eff10121d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:22:04.021595 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeeb608-b175-4258-89fa-bf2eff10121d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:04.022448 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeeb608-b175-4258-89fa-bf2eff10121d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:04.025003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a48f66-8d46-4cff-a665-34c00eb180ce None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:04.026383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-40a48f66-8d46-4cff-a665-34c00eb180ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:04.038501 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1244/2506] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:22:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:22:04.039799 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a48f66-8d46-4cff-a665-34c00eb180ce None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:04.042972 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeeb608-b175-4258-89fa-bf2eff10121d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:04.045083 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5fabab-d6a9-4ac4-9874-7b55d7f95a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:04.045858 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fabab-d6a9-4ac4-9874-7b55d7f95a30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:04.046140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fabab-d6a9-4ac4-9874-7b55d7f95a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:04.046568 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fabab-d6a9-4ac4-9874-7b55d7f95a30 admin admin] PATH_INFO: `/v3/users/8dded5e9c458496185063177ca60ba1f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:04.047719 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5fabab-d6a9-4ac4-9874-7b55d7f95a30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dded5e9c458496185063177ca60ba1f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:04.049651 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5fabab-d6a9-4ac4-9874-7b55d7f95a30 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:04.053685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeeb608-b175-4258-89fa-bf2eff10121d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:04.058477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e82650-d8e1-4791-a3c2-efc016f57b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:04.059071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e82650-d8e1-4791-a3c2-efc016f57b25 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:04.059348 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e82650-d8e1-4791-a3c2-efc016f57b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:04.059616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e82650-d8e1-4791-a3c2-efc016f57b25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:04.060260 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e82650-d8e1-4791-a3c2-efc016f57b25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:04.062145 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e82650-d8e1-4791-a3c2-efc016f57b25 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:04.199718 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1245/2507] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:22:04 2022] POST /identity/v3/projects => generated 359 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:22:04.204162 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e82650-d8e1-4791-a3c2-efc016f57b25 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:04.253301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ee5fabab-d6a9-4ac4-9874-7b55d7f95a30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dded5e9c458496185063177ca60ba1f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:04.267145 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e82650-d8e1-4791-a3c2-efc016f57b25 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:04.272500 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226987c8-1dff-4baf-96c9-5e43e9dc6713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:04.273087 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226987c8-1dff-4baf-96c9-5e43e9dc6713 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:04.273361 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226987c8-1dff-4baf-96c9-5e43e9dc6713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:04.273625 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226987c8-1dff-4baf-96c9-5e43e9dc6713 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:04.274235 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226987c8-1dff-4baf-96c9-5e43e9dc6713 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:04.276166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226987c8-1dff-4baf-96c9-5e43e9dc6713 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:04.312449 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ee5fabab-d6a9-4ac4-9874-7b55d7f95a30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dded5e9c458496185063177ca60ba1f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:04.319053 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:04.321230 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ee5fabab-d6a9-4ac4-9874-7b55d7f95a30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dded5e9c458496185063177ca60ba1f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:04.333010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ee5fabab-d6a9-4ac4-9874-7b55d7f95a30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dded5e9c458496185063177ca60ba1f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:04.334426 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ee5fabab-d6a9-4ac4-9874-7b55d7f95a30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dded5e9c458496185063177ca60ba1f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:04.341417 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1263/2508] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:22:04 2022] DELETE /identity/v3/users/8dded5e9c458496185063177ca60ba1f => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:22:04.345183 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1246/2509] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:22:04 2022] POST /identity/v3/users => generated 280 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:22:04.346048 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226987c8-1dff-4baf-96c9-5e43e9dc6713 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:04.346624 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7fa93-5b74-44f7-a127-2a0c1c382963 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:04.347132 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7fa93-5b74-44f7-a127-2a0c1c382963 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:04.347411 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7fa93-5b74-44f7-a127-2a0c1c382963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:04.347803 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-226987c8-1dff-4baf-96c9-5e43e9dc6713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:04.360256 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226987c8-1dff-4baf-96c9-5e43e9dc6713 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:04.366123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f2d7fa93-5b74-44f7-a127-2a0c1c382963 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:04.367501 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f2d7fa93-5b74-44f7-a127-2a0c1c382963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:04.370073 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:04.418254 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79c9e6-82e0-4713-9dd9-17a84f9b09de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:04.419113 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79c9e6-82e0-4713-9dd9-17a84f9b09de admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:04.419389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79c9e6-82e0-4713-9dd9-17a84f9b09de admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:04.419661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79c9e6-82e0-4713-9dd9-17a84f9b09de admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:04.420305 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e79c9e6-82e0-4713-9dd9-17a84f9b09de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:04.421939 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e79c9e6-82e0-4713-9dd9-17a84f9b09de admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:04.431649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1264/2510] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:22:04.432106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1247/2511] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:04.436215 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d7fa93-5b74-44f7-a127-2a0c1c382963 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:04.441948 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79c9e6-82e0-4713-9dd9-17a84f9b09de admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:04.444398 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4e79c9e6-82e0-4713-9dd9-17a84f9b09de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:04.446307 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d7fa93-5b74-44f7-a127-2a0c1c382963 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:04.464507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dedbad-46c9-4a06-a053-21086115cd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:04.464507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dedbad-46c9-4a06-a053-21086115cd70 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:04.464507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dedbad-46c9-4a06-a053-21086115cd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:04.464507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dedbad-46c9-4a06-a053-21086115cd70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:04.464991 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dedbad-46c9-4a06-a053-21086115cd70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:04.464991 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dedbad-46c9-4a06-a053-21086115cd70 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:04.465607 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1265/2512] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:22:04.469814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dedbad-46c9-4a06-a053-21086115cd70 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:04.479417 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dedbad-46c9-4a06-a053-21086115cd70 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:04.484123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e989422-3026-42b5-aa6a-776859893a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:04.484761 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e989422-3026-42b5-aa6a-776859893a91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:04.485041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e989422-3026-42b5-aa6a-776859893a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:04.487076 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e989422-3026-42b5-aa6a-776859893a91 admin admin] PATH_INFO: `/v3/projects/119c78f588da47fc92abda511c85d643/users/9b7b172a77f0459cabce47e88757373e/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:04.506549 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e989422-3026-42b5-aa6a-776859893a91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=119c78f588da47fc92abda511c85d643, user_id=9b7b172a77f0459cabce47e88757373e, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:04.508531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79c9e6-82e0-4713-9dd9-17a84f9b09de admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:04.509563 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e989422-3026-42b5-aa6a-776859893a91 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:04.512435 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb4a77-7520-4015-9b63-8171911bedc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:04.515815 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb4a77-7520-4015-9b63-8171911bedc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:04.515815 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb4a77-7520-4015-9b63-8171911bedc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:04.515815 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb4a77-7520-4015-9b63-8171911bedc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:04.523881 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb4a77-7520-4015-9b63-8171911bedc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:04.525295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb4a77-7520-4015-9b63-8171911bedc0 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:04.534942 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1248/2513] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:22:04.634566 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1266/2514] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:04 2022] PUT /identity/v3/projects/119c78f588da47fc92abda511c85d643/users/9b7b172a77f0459cabce47e88757373e/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:04.644962 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c55969-4525-4c1a-925c-a1a33fb909e2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:04.645245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c55969-4525-4c1a-925c-a1a33fb909e2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:04.645518 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c55969-4525-4c1a-925c-a1a33fb909e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:04.675145 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-89c55969-4525-4c1a-925c-a1a33fb909e2 None None] MFA Rules not processed for user `9b7b172a77f0459cabce47e88757373e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:04.676384 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-89c55969-4525-4c1a-925c-a1a33fb909e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:04.743541 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1249/2515] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:04 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:04.783140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e989422-3026-42b5-aa6a-776859893a91 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:04.784967 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4e989422-3026-42b5-aa6a-776859893a91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:04.846086 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e989422-3026-42b5-aa6a-776859893a91 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:04.850338 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89af63-1dc6-4b80-8cf3-1e6252a1acf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:04.850980 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89af63-1dc6-4b80-8cf3-1e6252a1acf1 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:04.851255 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89af63-1dc6-4b80-8cf3-1e6252a1acf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:04.851531 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89af63-1dc6-4b80-8cf3-1e6252a1acf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:04.860380 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce89af63-1dc6-4b80-8cf3-1e6252a1acf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:04.863304 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce89af63-1dc6-4b80-8cf3-1e6252a1acf1 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:04.871872 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1267/2516] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:04 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:22:04.890608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c55969-4525-4c1a-925c-a1a33fb909e2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:04.935269 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:04.953283 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c55969-4525-4c1a-925c-a1a33fb909e2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:04.957967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c19d1-82f2-4968-8ff8-74a82e8a08d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:04.959121 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c19d1-82f2-4968-8ff8-74a82e8a08d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:04.959418 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c19d1-82f2-4968-8ff8-74a82e8a08d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:04.959693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c19d1-82f2-4968-8ff8-74a82e8a08d8 admin admin] PATH_INFO: `/v3/users/9b7b172a77f0459cabce47e88757373e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:04.960835 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c19d1-82f2-4968-8ff8-74a82e8a08d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b7b172a77f0459cabce47e88757373e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:04.962793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c19d1-82f2-4968-8ff8-74a82e8a08d8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:04.995478 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89af63-1dc6-4b80-8cf3-1e6252a1acf1 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:04.996820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ce89af63-1dc6-4b80-8cf3-1e6252a1acf1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:05.012331 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89af63-1dc6-4b80-8cf3-1e6252a1acf1 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:05.017290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e190b3-b6cb-4138-a023-153a3226e26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:05.017954 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e190b3-b6cb-4138-a023-153a3226e26a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:05.018215 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e190b3-b6cb-4138-a023-153a3226e26a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:05.019080 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e190b3-b6cb-4138-a023-153a3226e26a admin admin] PATH_INFO: `/v3/projects/242c8af510fe476085e5766b88085283` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:05.019834 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e190b3-b6cb-4138-a023-153a3226e26a admin admin] RBAC: Authorizing `identity:delete_project(project_id=242c8af510fe476085e5766b88085283)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:05.021802 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e190b3-b6cb-4138-a023-153a3226e26a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:05.160363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:05.167526 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c3e190b3-b6cb-4138-a023-153a3226e26a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '242c8af510fe476085e5766b88085283', 'request_id': 'req-c3e190b3-b6cb-4138-a023-153a3226e26a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:05.167697 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1f4c19d1-82f2-4968-8ff8-74a82e8a08d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b7b172a77f0459cabce47e88757373e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:05.173996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c3e190b3-b6cb-4138-a023-153a3226e26a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '242c8af510fe476085e5766b88085283', 'request_id': 'req-c3e190b3-b6cb-4138-a023-153a3226e26a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:05.415415 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1f4c19d1-82f2-4968-8ff8-74a82e8a08d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b7b172a77f0459cabce47e88757373e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:05.429556 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1f4c19d1-82f2-4968-8ff8-74a82e8a08d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b7b172a77f0459cabce47e88757373e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:05.430380 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1f4c19d1-82f2-4968-8ff8-74a82e8a08d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b7b172a77f0459cabce47e88757373e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:05.431266 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1f4c19d1-82f2-4968-8ff8-74a82e8a08d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b7b172a77f0459cabce47e88757373e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:05.445077 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1250/2517] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:22:04 2022] DELETE /identity/v3/users/9b7b172a77f0459cabce47e88757373e => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:05.449599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bb92f-f6ad-4bc4-b4f4-d4e827c3e55a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:05.449880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bb92f-f6ad-4bc4-b4f4-d4e827c3e55a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:05.450152 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bb92f-f6ad-4bc4-b4f4-d4e827c3e55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:05.491349 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-183bb92f-f6ad-4bc4-b4f4-d4e827c3e55a None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:05.492661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-183bb92f-f6ad-4bc4-b4f4-d4e827c3e55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:05.501120 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c3e190b3-b6cb-4138-a023-153a3226e26a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '242c8af510fe476085e5766b88085283', 'request_id': 'req-c3e190b3-b6cb-4138-a023-153a3226e26a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:05.512869 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c3e190b3-b6cb-4138-a023-153a3226e26a admin admin] The token cache is being invalidate because project 242c8af510fe476085e5766b88085283 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:22:05.516021 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c3e190b3-b6cb-4138-a023-153a3226e26a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:05.516021 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c3e190b3-b6cb-4138-a023-153a3226e26a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:05.516021 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1268/2518] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:22:04 2022] DELETE /identity/v3/projects/242c8af510fe476085e5766b88085283 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:22:05.519418 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e190b3-b6cb-4138-a023-153a3226e26a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:05.520729 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c3e190b3-b6cb-4138-a023-153a3226e26a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:05.567545 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1251/2519] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:22:05.587413 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bb92f-f6ad-4bc4-b4f4-d4e827c3e55a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:05.588854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-183bb92f-f6ad-4bc4-b4f4-d4e827c3e55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:05.632913 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:05.641687 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e190b3-b6cb-4138-a023-153a3226e26a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:05.647362 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d7c64-a2a3-4605-a500-536fb69c8389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:05.648058 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d7c64-a2a3-4605-a500-536fb69c8389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:05.648530 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d7c64-a2a3-4605-a500-536fb69c8389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:05.648819 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d7c64-a2a3-4605-a500-536fb69c8389 admin admin] PATH_INFO: `/v3/users/f7bb80a0d246414c95ac72d55947ac2a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:05.649975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d7c64-a2a3-4605-a500-536fb69c8389 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7bb80a0d246414c95ac72d55947ac2a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:05.651954 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d7c64-a2a3-4605-a500-536fb69c8389 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:05.660150 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bb92f-f6ad-4bc4-b4f4-d4e827c3e55a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:05.664821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865bb15-2d59-4a72-8f9c-9fae95c16bd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:05.665481 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865bb15-2d59-4a72-8f9c-9fae95c16bd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:05.665710 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865bb15-2d59-4a72-8f9c-9fae95c16bd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:05.665973 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865bb15-2d59-4a72-8f9c-9fae95c16bd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:05.678058 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865bb15-2d59-4a72-8f9c-9fae95c16bd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:05.678058 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865bb15-2d59-4a72-8f9c-9fae95c16bd2 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:05.687285 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1252/2520] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:22:05.959596 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-302d7c64-a2a3-4605-a500-536fb69c8389 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7bb80a0d246414c95ac72d55947ac2a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:06.078677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-302d7c64-a2a3-4605-a500-536fb69c8389 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7bb80a0d246414c95ac72d55947ac2a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:06.086535 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-302d7c64-a2a3-4605-a500-536fb69c8389 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7bb80a0d246414c95ac72d55947ac2a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:06.098529 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-302d7c64-a2a3-4605-a500-536fb69c8389 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7bb80a0d246414c95ac72d55947ac2a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:06.099817 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-302d7c64-a2a3-4605-a500-536fb69c8389 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7bb80a0d246414c95ac72d55947ac2a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:06.106003 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1269/2521] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:22:05 2022] DELETE /identity/v3/users/f7bb80a0d246414c95ac72d55947ac2a => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:22:06.217442 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865bb15-2d59-4a72-8f9c-9fae95c16bd2 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:06.219733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6865bb15-2d59-4a72-8f9c-9fae95c16bd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:06.233238 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865bb15-2d59-4a72-8f9c-9fae95c16bd2 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:06.297542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583386b-02b9-4833-95a7-d2e7aefafba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:06.298166 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583386b-02b9-4833-95a7-d2e7aefafba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:06.298475 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583386b-02b9-4833-95a7-d2e7aefafba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:06.298804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583386b-02b9-4833-95a7-d2e7aefafba8 admin admin] PATH_INFO: `/v3/projects/119c78f588da47fc92abda511c85d643` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:06.300028 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583386b-02b9-4833-95a7-d2e7aefafba8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=119c78f588da47fc92abda511c85d643)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:06.301932 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583386b-02b9-4833-95a7-d2e7aefafba8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:06.422618 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f583386b-02b9-4833-95a7-d2e7aefafba8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '119c78f588da47fc92abda511c85d643', 'request_id': 'req-f583386b-02b9-4833-95a7-d2e7aefafba8'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:06.546661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f583386b-02b9-4833-95a7-d2e7aefafba8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '119c78f588da47fc92abda511c85d643', 'request_id': 'req-f583386b-02b9-4833-95a7-d2e7aefafba8'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:06.553581 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f583386b-02b9-4833-95a7-d2e7aefafba8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '119c78f588da47fc92abda511c85d643', 'request_id': 'req-f583386b-02b9-4833-95a7-d2e7aefafba8'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:06.563941 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f583386b-02b9-4833-95a7-d2e7aefafba8 admin admin] The token cache is being invalidate because project 119c78f588da47fc92abda511c85d643 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:22:06.564225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f583386b-02b9-4833-95a7-d2e7aefafba8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:06.564856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f583386b-02b9-4833-95a7-d2e7aefafba8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:06.566334 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1253/2522] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:22:06 2022] DELETE /identity/v3/projects/119c78f588da47fc92abda511c85d643 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:06.571844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc3a87-52c3-45b2-8d11-2b8de05fe152 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:06.572307 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc3a87-52c3-45b2-8d11-2b8de05fe152 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:06.572412 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc3a87-52c3-45b2-8d11-2b8de05fe152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:06.593589 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:06.628493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583386b-02b9-4833-95a7-d2e7aefafba8 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:06.629890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f583386b-02b9-4833-95a7-d2e7aefafba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:06.635385 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4efc3a87-52c3-45b2-8d11-2b8de05fe152 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:06.636598 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4efc3a87-52c3-45b2-8d11-2b8de05fe152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:06.702985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1270/2523] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:06.709468 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583386b-02b9-4833-95a7-d2e7aefafba8 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:06.719423 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc58b6-31f6-47b9-803a-26c1f32af987 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:06.719423 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc58b6-31f6-47b9-803a-26c1f32af987 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:06.719423 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc58b6-31f6-47b9-803a-26c1f32af987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:06.719772 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4405b6-5279-4634-ac56-1b99358f5059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:06.719772 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4405b6-5279-4634-ac56-1b99358f5059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:06.719772 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4405b6-5279-4634-ac56-1b99358f5059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:06.719772 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4405b6-5279-4634-ac56-1b99358f5059 admin admin] PATH_INFO: `/v3/projects/2fbe3198d4034a7b867dc0ed33629d64` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:06.720151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4405b6-5279-4634-ac56-1b99358f5059 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fbe3198d4034a7b867dc0ed33629d64)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:06.720151 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4405b6-5279-4634-ac56-1b99358f5059 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:06.722132 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:06.737883 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-53cc58b6-31f6-47b9-803a-26c1f32af987 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:06.737883 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-53cc58b6-31f6-47b9-803a-26c1f32af987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:06.743665 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1271/2524] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:22:06.747953 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc58b6-31f6-47b9-803a-26c1f32af987 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:06.757894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc58b6-31f6-47b9-803a-26c1f32af987 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:06.765456 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2aaad-f28b-4d96-a4dd-d19b6265e4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:06.765456 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2aaad-f28b-4d96-a4dd-d19b6265e4fb admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:06.765456 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2aaad-f28b-4d96-a4dd-d19b6265e4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:06.765456 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2aaad-f28b-4d96-a4dd-d19b6265e4fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:06.765883 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2aaad-f28b-4d96-a4dd-d19b6265e4fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:22:06.765883 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2aaad-f28b-4d96-a4dd-d19b6265e4fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:06.766194 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2aaad-f28b-4d96-a4dd-d19b6265e4fb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:06.778175 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1272/2525] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 17:22:06.782560 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2aaad-f28b-4d96-a4dd-d19b6265e4fb admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:06.791919 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2aaad-f28b-4d96-a4dd-d19b6265e4fb admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:06.799876 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f3415-b782-4297-9855-cc1ca7f700a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:06.799876 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f3415-b782-4297-9855-cc1ca7f700a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:06.799876 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f3415-b782-4297-9855-cc1ca7f700a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:06.799876 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f3415-b782-4297-9855-cc1ca7f700a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:06.800365 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f3415-b782-4297-9855-cc1ca7f700a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:06.800365 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f3415-b782-4297-9855-cc1ca7f700a1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:06.900500 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1273/2526] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:22:06 2022] POST /identity/v3/projects => generated 369 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:22:06.901597 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f3415-b782-4297-9855-cc1ca7f700a1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:06.908440 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8f4405b6-5279-4634-ac56-1b99358f5059 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fbe3198d4034a7b867dc0ed33629d64', 'request_id': 'req-8f4405b6-5279-4634-ac56-1b99358f5059'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:06.964299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f3415-b782-4297-9855-cc1ca7f700a1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:06.969214 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca76d0-6f51-4b6b-96bd-d8af0f28cfdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:06.969801 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca76d0-6f51-4b6b-96bd-d8af0f28cfdf admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:06.970074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca76d0-6f51-4b6b-96bd-d8af0f28cfdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:06.970336 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca76d0-6f51-4b6b-96bd-d8af0f28cfdf admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:06.970974 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca76d0-6f51-4b6b-96bd-d8af0f28cfdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:06.972915 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca76d0-6f51-4b6b-96bd-d8af0f28cfdf admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:07.009595 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8f4405b6-5279-4634-ac56-1b99358f5059 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fbe3198d4034a7b867dc0ed33629d64', 'request_id': 'req-8f4405b6-5279-4634-ac56-1b99358f5059'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:07.018241 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8f4405b6-5279-4634-ac56-1b99358f5059 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fbe3198d4034a7b867dc0ed33629d64', 'request_id': 'req-8f4405b6-5279-4634-ac56-1b99358f5059'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:07.031025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8f4405b6-5279-4634-ac56-1b99358f5059 admin admin] The token cache is being invalidate because project 2fbe3198d4034a7b867dc0ed33629d64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:22:07.031311 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8f4405b6-5279-4634-ac56-1b99358f5059 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:07.031972 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8f4405b6-5279-4634-ac56-1b99358f5059 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:07.033480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1254/2527] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:22:06 2022] DELETE /identity/v3/projects/2fbe3198d4034a7b867dc0ed33629d64 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:07.038558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894e7d6-23d2-4243-8383-5fb1f5912c50 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:07.041771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894e7d6-23d2-4243-8383-5fb1f5912c50 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:07.041771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894e7d6-23d2-4243-8383-5fb1f5912c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:07.059005 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7894e7d6-23d2-4243-8383-5fb1f5912c50 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:07.060201 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7894e7d6-23d2-4243-8383-5fb1f5912c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:07.067960 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1255/2528] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:07.076375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d95166-4af4-4e67-9fce-b721dd0aebf3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:07.079835 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d95166-4af4-4e67-9fce-b721dd0aebf3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:07.079835 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d95166-4af4-4e67-9fce-b721dd0aebf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:07.095266 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-03d95166-4af4-4e67-9fce-b721dd0aebf3 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:07.096433 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-03d95166-4af4-4e67-9fce-b721dd0aebf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:07.104177 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1256/2529] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:22:07.108337 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1274/2530] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:22:06 2022] POST /identity/v3/users => generated 285 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:22:07.108579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d95166-4af4-4e67-9fce-b721dd0aebf3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:07.118995 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca76d0-6f51-4b6b-96bd-d8af0f28cfdf admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:07.119174 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d95166-4af4-4e67-9fce-b721dd0aebf3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:07.120736 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-daca76d0-6f51-4b6b-96bd-d8af0f28cfdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:07.123874 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56faa151-806b-4728-99a0-14c770d5e973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:07.124472 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faa151-806b-4728-99a0-14c770d5e973 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:07.124749 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faa151-806b-4728-99a0-14c770d5e973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:07.125018 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faa151-806b-4728-99a0-14c770d5e973 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:07.125663 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56faa151-806b-4728-99a0-14c770d5e973 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:22:07.125966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56faa151-806b-4728-99a0-14c770d5e973 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:07.127678 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56faa151-806b-4728-99a0-14c770d5e973 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:07.133900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca76d0-6f51-4b6b-96bd-d8af0f28cfdf admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:07.138837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806a5e8-2e42-4a85-abcc-873d56e7458e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:07.139452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806a5e8-2e42-4a85-abcc-873d56e7458e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:07.139778 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806a5e8-2e42-4a85-abcc-873d56e7458e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:07.140045 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806a5e8-2e42-4a85-abcc-873d56e7458e admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:07.140753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806a5e8-2e42-4a85-abcc-873d56e7458e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:07.140968 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1257/2531] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 17:22:07.142289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806a5e8-2e42-4a85-abcc-873d56e7458e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:07.145880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56faa151-806b-4728-99a0-14c770d5e973 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:07.155570 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1275/2532] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:22:07.155756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56faa151-806b-4728-99a0-14c770d5e973 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:07.156564 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806a5e8-2e42-4a85-abcc-873d56e7458e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:07.160330 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec6b4a1-6471-4d7e-8bb5-a1a45a38a9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:07.160901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6b4a1-6471-4d7e-8bb5-a1a45a38a9b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:07.161176 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6b4a1-6471-4d7e-8bb5-a1a45a38a9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:07.161440 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6b4a1-6471-4d7e-8bb5-a1a45a38a9b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:07.162071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec6b4a1-6471-4d7e-8bb5-a1a45a38a9b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:07.163991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec6b4a1-6471-4d7e-8bb5-a1a45a38a9b9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:07.166183 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806a5e8-2e42-4a85-abcc-873d56e7458e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:07.171320 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744f76c-7dd8-4d8a-bb15-8baa6c3427b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:07.171909 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744f76c-7dd8-4d8a-bb15-8baa6c3427b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:07.172182 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744f76c-7dd8-4d8a-bb15-8baa6c3427b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:07.172450 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744f76c-7dd8-4d8a-bb15-8baa6c3427b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:07.173080 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744f76c-7dd8-4d8a-bb15-8baa6c3427b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:07.174700 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744f76c-7dd8-4d8a-bb15-8baa6c3427b5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:07.186443 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1276/2533] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:07.191455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744f76c-7dd8-4d8a-bb15-8baa6c3427b5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:07.201923 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744f76c-7dd8-4d8a-bb15-8baa6c3427b5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:07.206541 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6474d-57e1-4cea-96c4-18f4f9306fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:07.207183 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6474d-57e1-4cea-96c4-18f4f9306fd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:07.207463 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6474d-57e1-4cea-96c4-18f4f9306fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:07.207738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6474d-57e1-4cea-96c4-18f4f9306fd6 admin admin] PATH_INFO: `/v3/projects/2c9ea9542e1b4d95b9af64192e8544f1/users/0897fe87870e464cab17d9592786c6d8/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:07.222383 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c6474d-57e1-4cea-96c4-18f4f9306fd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c9ea9542e1b4d95b9af64192e8544f1, user_id=0897fe87870e464cab17d9592786c6d8, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:07.225104 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c6474d-57e1-4cea-96c4-18f4f9306fd6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:07.283966 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1258/2534] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:22:07 2022] POST /identity/v3/projects => generated 387 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:22:07.288903 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec6b4a1-6471-4d7e-8bb5-a1a45a38a9b9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:07.313870 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1277/2535] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:07 2022] PUT /identity/v3/projects/2c9ea9542e1b4d95b9af64192e8544f1/users/0897fe87870e464cab17d9592786c6d8/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:22:07.324899 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510b0eb-3cc2-46d7-9bf4-a7f7e331c799 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:07.325180 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510b0eb-3cc2-46d7-9bf4-a7f7e331c799 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:07.325505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510b0eb-3cc2-46d7-9bf4-a7f7e331c799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:07.349729 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec6b4a1-6471-4d7e-8bb5-a1a45a38a9b9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:07.353456 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c510b0eb-3cc2-46d7-9bf4-a7f7e331c799 None None] MFA Rules not processed for user `0897fe87870e464cab17d9592786c6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:07.354709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c510b0eb-3cc2-46d7-9bf4-a7f7e331c799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:07.410208 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d4340-6ad6-40b5-9461-29e57c6f0c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:07.410851 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d4340-6ad6-40b5-9461-29e57c6f0c07 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:07.411127 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d4340-6ad6-40b5-9461-29e57c6f0c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:07.411403 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d4340-6ad6-40b5-9461-29e57c6f0c07 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:07.412077 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d4340-6ad6-40b5-9461-29e57c6f0c07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:07.414787 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d4340-6ad6-40b5-9461-29e57c6f0c07 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:07.420668 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1278/2536] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:22:07.433955 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510b0eb-3cc2-46d7-9bf4-a7f7e331c799 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:07.435369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c510b0eb-3cc2-46d7-9bf4-a7f7e331c799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:07.509112 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510b0eb-3cc2-46d7-9bf4-a7f7e331c799 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:07.512748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67533592-ad7e-4ea7-9888-393d7d49f1ee service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10f4f54890624da29ff217081445b6bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:07.513347 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67533592-ad7e-4ea7-9888-393d7d49f1ee service glance] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:07.513621 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67533592-ad7e-4ea7-9888-393d7d49f1ee service glance] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:07.513912 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67533592-ad7e-4ea7-9888-393d7d49f1ee service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:07.522967 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67533592-ad7e-4ea7-9888-393d7d49f1ee service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:07.524491 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67533592-ad7e-4ea7-9888-393d7d49f1ee service glance] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:07.533649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1279/2537] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:22:07.549752 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1259/2538] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:22:07 2022] POST /identity/v3/users => generated 294 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:22:07.550937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67533592-ad7e-4ea7-9888-393d7d49f1ee service glance] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:07.565094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67533592-ad7e-4ea7-9888-393d7d49f1ee service glance] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:07.570271 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc998f-f9e1-478b-8cc6-ecf99b9c5ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:07.570905 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc998f-f9e1-478b-8cc6-ecf99b9c5ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:07.571186 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc998f-f9e1-478b-8cc6-ecf99b9c5ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:07.571456 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc998f-f9e1-478b-8cc6-ecf99b9c5ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:07.572091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc998f-f9e1-478b-8cc6-ecf99b9c5ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:07.573803 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc998f-f9e1-478b-8cc6-ecf99b9c5ef6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:07.585532 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1280/2539] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:07.589493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d4340-6ad6-40b5-9461-29e57c6f0c07 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:07.599726 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d4340-6ad6-40b5-9461-29e57c6f0c07 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:07.612284 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1eccbb-05b5-480a-967d-43dd41337f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:07.612284 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1eccbb-05b5-480a-967d-43dd41337f9b admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:07.612284 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1eccbb-05b5-480a-967d-43dd41337f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:07.612284 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1eccbb-05b5-480a-967d-43dd41337f9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:07.612613 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1eccbb-05b5-480a-967d-43dd41337f9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:07.612613 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1eccbb-05b5-480a-967d-43dd41337f9b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:07.622122 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1260/2540] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:07.626298 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc998f-f9e1-478b-8cc6-ecf99b9c5ef6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:07.637434 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc998f-f9e1-478b-8cc6-ecf99b9c5ef6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:07.642862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb1c02-9ad0-4390-84d3-f8be45d00cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:07.651701 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb1c02-9ad0-4390-84d3-f8be45d00cb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:07.651701 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb1c02-9ad0-4390-84d3-f8be45d00cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:07.651701 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb1c02-9ad0-4390-84d3-f8be45d00cb3 admin admin] PATH_INFO: `/v3/projects/c8b2c5fa7a5847908708c70ae3352fed/users/182b1efb715147b095fce63a2d9b0ea6/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:07.660899 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb1c02-9ad0-4390-84d3-f8be45d00cb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8b2c5fa7a5847908708c70ae3352fed, user_id=182b1efb715147b095fce63a2d9b0ea6, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:07.665043 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb1c02-9ad0-4390-84d3-f8be45d00cb3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:07.717847 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:07.873171 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1281/2541] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:07 2022] PUT /identity/v3/projects/c8b2c5fa7a5847908708c70ae3352fed/users/182b1efb715147b095fce63a2d9b0ea6/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:07.883923 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517da04-5efb-4e95-8fd4-3be864298fee None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:07.885519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517da04-5efb-4e95-8fd4-3be864298fee None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:07.885519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517da04-5efb-4e95-8fd4-3be864298fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:07.914112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0517da04-5efb-4e95-8fd4-3be864298fee None None] MFA Rules not processed for user `182b1efb715147b095fce63a2d9b0ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:07.915462 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0517da04-5efb-4e95-8fd4-3be864298fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:07.981273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1261/2542] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:07.985796 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb1c02-9ad0-4390-84d3-f8be45d00cb3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:07.998964 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:08.048463 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb1c02-9ad0-4390-84d3-f8be45d00cb3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:08.053008 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b8a3c4-bd19-45d6-b68b-f9cef47e2e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:08.053720 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b8a3c4-bd19-45d6-b68b-f9cef47e2e35 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:08.054045 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b8a3c4-bd19-45d6-b68b-f9cef47e2e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:08.054327 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b8a3c4-bd19-45d6-b68b-f9cef47e2e35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:08.054994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b8a3c4-bd19-45d6-b68b-f9cef47e2e35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:08.056909 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b8a3c4-bd19-45d6-b68b-f9cef47e2e35 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:08.205329 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1282/2543] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:22:07 2022] POST /identity/v3/projects => generated 387 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:22:08.209387 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517da04-5efb-4e95-8fd4-3be864298fee None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:08.220449 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517da04-5efb-4e95-8fd4-3be864298fee None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:08.279892 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7249b0-ec4c-402e-b75f-f8bb18c5b872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:08.280498 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7249b0-ec4c-402e-b75f-f8bb18c5b872 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:08.280769 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7249b0-ec4c-402e-b75f-f8bb18c5b872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:08.281041 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7249b0-ec4c-402e-b75f-f8bb18c5b872 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:08.281678 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7249b0-ec4c-402e-b75f-f8bb18c5b872 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:08.283605 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7249b0-ec4c-402e-b75f-f8bb18c5b872 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:08.330295 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:08.398771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1262/2544] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:22:08 2022] POST /identity/v3/users => generated 294 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:22:08.403340 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b8a3c4-bd19-45d6-b68b-f9cef47e2e35 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:08.413418 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b8a3c4-bd19-45d6-b68b-f9cef47e2e35 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:08.418427 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a60f2-9967-4534-94b8-f6d7c835a0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:08.419025 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a60f2-9967-4534-94b8-f6d7c835a0ea admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:08.419386 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a60f2-9967-4534-94b8-f6d7c835a0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:08.419658 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a60f2-9967-4534-94b8-f6d7c835a0ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:08.420299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a60f2-9967-4534-94b8-f6d7c835a0ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:08.421900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a60f2-9967-4534-94b8-f6d7c835a0ea admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:08.434530 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1283/2545] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:08.438662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7249b0-ec4c-402e-b75f-f8bb18c5b872 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:08.449779 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7249b0-ec4c-402e-b75f-f8bb18c5b872 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:08.454673 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc574c2-4a1a-46e2-a6c3-03d729082aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:08.455243 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc574c2-4a1a-46e2-a6c3-03d729082aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:08.455551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc574c2-4a1a-46e2-a6c3-03d729082aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:08.455823 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc574c2-4a1a-46e2-a6c3-03d729082aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:08.456469 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc574c2-4a1a-46e2-a6c3-03d729082aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:08.458123 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc574c2-4a1a-46e2-a6c3-03d729082aa1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:08.472138 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1263/2546] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:08.473538 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a60f2-9967-4534-94b8-f6d7c835a0ea admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:08.483906 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a60f2-9967-4534-94b8-f6d7c835a0ea admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:08.488664 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1c47f-32bf-4fa4-b4b5-fc546ada1e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:08.489310 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1c47f-32bf-4fa4-b4b5-fc546ada1e48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:08.489707 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1c47f-32bf-4fa4-b4b5-fc546ada1e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:08.490023 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1c47f-32bf-4fa4-b4b5-fc546ada1e48 admin admin] PATH_INFO: `/v3/projects/6335692f6bbf4abc84cf042d8a40c01f/users/9bba4c4beeeb44a5ab5cff7456b04c72/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:08.509432 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1c47f-32bf-4fa4-b4b5-fc546ada1e48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6335692f6bbf4abc84cf042d8a40c01f, user_id=9bba4c4beeeb44a5ab5cff7456b04c72, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:08.512508 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1c47f-32bf-4fa4-b4b5-fc546ada1e48 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:08.672544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1284/2547] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:08 2022] PUT /identity/v3/projects/6335692f6bbf4abc84cf042d8a40c01f/users/9bba4c4beeeb44a5ab5cff7456b04c72/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:22:08.684711 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834107c-4ce8-4673-8301-ce8f8706ce0d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:08.684996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834107c-4ce8-4673-8301-ce8f8706ce0d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:08.685273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834107c-4ce8-4673-8301-ce8f8706ce0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:08.715343 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6834107c-4ce8-4673-8301-ce8f8706ce0d None None] MFA Rules not processed for user `9bba4c4beeeb44a5ab5cff7456b04c72`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:08.716570 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6834107c-4ce8-4673-8301-ce8f8706ce0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:08.744427 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:08.783892 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1264/2548] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:22:08.802382 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1c47f-32bf-4fa4-b4b5-fc546ada1e48 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:08.865813 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1c47f-32bf-4fa4-b4b5-fc546ada1e48 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:08.871231 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524f693-75df-43a2-aa75-4e72ba0ca0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:08.871836 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524f693-75df-43a2-aa75-4e72ba0ca0f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:08.872142 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524f693-75df-43a2-aa75-4e72ba0ca0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:08.872434 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524f693-75df-43a2-aa75-4e72ba0ca0f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:08.873165 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524f693-75df-43a2-aa75-4e72ba0ca0f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:08.875176 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524f693-75df-43a2-aa75-4e72ba0ca0f9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:09.012869 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1285/2549] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:22:08 2022] POST /identity/v3/projects => generated 387 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:22:09.016811 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6834107c-4ce8-4673-8301-ce8f8706ce0d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:09.027561 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6834107c-4ce8-4673-8301-ce8f8706ce0d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:09.085418 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37f421-e628-45a6-9457-8e02448f2c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:09.086026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37f421-e628-45a6-9457-8e02448f2c7c admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:09.086306 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37f421-e628-45a6-9457-8e02448f2c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:09.086606 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37f421-e628-45a6-9457-8e02448f2c7c admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:09.087229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f37f421-e628-45a6-9457-8e02448f2c7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:09.089170 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f37f421-e628-45a6-9457-8e02448f2c7c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:09.256426 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1265/2550] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:22:09 2022] POST /identity/v3/users => generated 300 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:22:09.260993 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524f693-75df-43a2-aa75-4e72ba0ca0f9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:09.271326 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524f693-75df-43a2-aa75-4e72ba0ca0f9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:09.277692 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809002a-ef61-4a88-bdcf-08f0e6cbeeb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:09.278421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809002a-ef61-4a88-bdcf-08f0e6cbeeb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:09.280863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809002a-ef61-4a88-bdcf-08f0e6cbeeb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:09.280863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809002a-ef61-4a88-bdcf-08f0e6cbeeb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:09.280863 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6809002a-ef61-4a88-bdcf-08f0e6cbeeb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:09.281539 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6809002a-ef61-4a88-bdcf-08f0e6cbeeb5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:09.294531 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1286/2551] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:09.299001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37f421-e628-45a6-9457-8e02448f2c7c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:09.309992 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37f421-e628-45a6-9457-8e02448f2c7c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:09.314675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19328887-f67b-44cb-ad17-5d6b4250f52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:09.315323 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19328887-f67b-44cb-ad17-5d6b4250f52d admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:09.315601 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19328887-f67b-44cb-ad17-5d6b4250f52d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:09.315930 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19328887-f67b-44cb-ad17-5d6b4250f52d admin admin] PATH_INFO: `/v3/projects/282f4a5318164e2e938c8988b69cdd50/users/1d467af2347b4817ba70bcc454f2eee7/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:09.333840 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19328887-f67b-44cb-ad17-5d6b4250f52d admin admin] RBAC: Authorizing `identity:create_grant(project_id=282f4a5318164e2e938c8988b69cdd50, user_id=1d467af2347b4817ba70bcc454f2eee7, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:09.337311 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19328887-f67b-44cb-ad17-5d6b4250f52d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:09.468315 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1266/2552] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:09 2022] PUT /identity/v3/projects/282f4a5318164e2e938c8988b69cdd50/users/1d467af2347b4817ba70bcc454f2eee7/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:09.472464 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809002a-ef61-4a88-bdcf-08f0e6cbeeb5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:09.483942 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809002a-ef61-4a88-bdcf-08f0e6cbeeb5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:09.538140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:09.547626 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379f77b-ab1a-444a-8073-34c252d31951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:09.548239 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379f77b-ab1a-444a-8073-34c252d31951 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:09.548518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379f77b-ab1a-444a-8073-34c252d31951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:09.548789 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379f77b-ab1a-444a-8073-34c252d31951 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:09.549429 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379f77b-ab1a-444a-8073-34c252d31951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:09.551100 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379f77b-ab1a-444a-8073-34c252d31951 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:09.561349 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1287/2553] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:09.566105 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19328887-f67b-44cb-ad17-5d6b4250f52d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:09.577378 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19328887-f67b-44cb-ad17-5d6b4250f52d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:09.582082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b713bc-08c3-48fa-bac5-f3dc1e696018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:09.582822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b713bc-08c3-48fa-bac5-f3dc1e696018 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:09.583119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b713bc-08c3-48fa-bac5-f3dc1e696018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:09.583390 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b713bc-08c3-48fa-bac5-f3dc1e696018 admin admin] PATH_INFO: `/v3/projects/282f4a5318164e2e938c8988b69cdd50/users/1d467af2347b4817ba70bcc454f2eee7/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:09.590054 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b713bc-08c3-48fa-bac5-f3dc1e696018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=282f4a5318164e2e938c8988b69cdd50, user_id=1d467af2347b4817ba70bcc454f2eee7, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:09.590054 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b713bc-08c3-48fa-bac5-f3dc1e696018 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:09.600624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1267/2554] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:09 2022] PUT /identity/v3/projects/282f4a5318164e2e938c8988b69cdd50/users/1d467af2347b4817ba70bcc454f2eee7/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:09.617541 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a080266-8123-42f2-bea6-f2ddde2cdbf2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:09.617845 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a080266-8123-42f2-bea6-f2ddde2cdbf2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:09.620758 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a080266-8123-42f2-bea6-f2ddde2cdbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:09.649753 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6a080266-8123-42f2-bea6-f2ddde2cdbf2 None None] MFA Rules not processed for user `1d467af2347b4817ba70bcc454f2eee7`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:09.651117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6a080266-8123-42f2-bea6-f2ddde2cdbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:09.721304 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1288/2555] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:09 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:09.738110 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b713bc-08c3-48fa-bac5-f3dc1e696018 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:09.739900 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-39b713bc-08c3-48fa-bac5-f3dc1e696018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:09.800318 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b713bc-08c3-48fa-bac5-f3dc1e696018 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:09.804763 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb6841-6da9-457a-8e53-c7b02ffe3631 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:09.805378 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb6841-6da9-457a-8e53-c7b02ffe3631 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:09.805653 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb6841-6da9-457a-8e53-c7b02ffe3631 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:09.805923 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb6841-6da9-457a-8e53-c7b02ffe3631 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:09.815493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb6841-6da9-457a-8e53-c7b02ffe3631 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:09.818108 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb6841-6da9-457a-8e53-c7b02ffe3631 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:09.826790 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1268/2556] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:09 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:22:11.214290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a080266-8123-42f2-bea6-f2ddde2cdbf2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:11.215797 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6a080266-8123-42f2-bea6-f2ddde2cdbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:11.227751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a080266-8123-42f2-bea6-f2ddde2cdbf2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:11.231326 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ace2f-3cf6-4fd2-a726-04fc1ea17638 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:11.231980 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ace2f-3cf6-4fd2-a726-04fc1ea17638 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:11.232193 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ace2f-3cf6-4fd2-a726-04fc1ea17638 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:11.232444 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ace2f-3cf6-4fd2-a726-04fc1ea17638 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:11.289519 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ace2f-3cf6-4fd2-a726-04fc1ea17638 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:11.291032 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ace2f-3cf6-4fd2-a726-04fc1ea17638 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:11.300406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1289/2557] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:11 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:22:16.443045 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb6841-6da9-457a-8e53-c7b02ffe3631 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:16.452075 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb6841-6da9-457a-8e53-c7b02ffe3631 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:16.455917 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e362b-7435-4854-803c-c0c7ed9efad4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:16.456515 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e362b-7435-4854-803c-c0c7ed9efad4 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:16.456805 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e362b-7435-4854-803c-c0c7ed9efad4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:16.457051 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e362b-7435-4854-803c-c0c7ed9efad4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:16.476725 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:16.520078 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e362b-7435-4854-803c-c0c7ed9efad4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:16.521508 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e362b-7435-4854-803c-c0c7ed9efad4 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:16.530991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1269/2558] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:16 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:22:21.282718 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ace2f-3cf6-4fd2-a726-04fc1ea17638 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:21.358647 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ace2f-3cf6-4fd2-a726-04fc1ea17638 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:21.363796 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76aa6bb-c391-4889-9610-1fdcf4f1128b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:21.364527 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76aa6bb-c391-4889-9610-1fdcf4f1128b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:21.364838 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76aa6bb-c391-4889-9610-1fdcf4f1128b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:21.365107 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76aa6bb-c391-4889-9610-1fdcf4f1128b admin admin] PATH_INFO: `/v3/users/182b1efb715147b095fce63a2d9b0ea6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:21.367225 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76aa6bb-c391-4889-9610-1fdcf4f1128b admin admin] RBAC: Authorizing `identity:delete_user(user_id=182b1efb715147b095fce63a2d9b0ea6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:21.369171 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76aa6bb-c391-4889-9610-1fdcf4f1128b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:21.393611 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:21.627613 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c76aa6bb-c391-4889-9610-1fdcf4f1128b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '182b1efb715147b095fce63a2d9b0ea6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:21.723720 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c76aa6bb-c391-4889-9610-1fdcf4f1128b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '182b1efb715147b095fce63a2d9b0ea6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:21.734815 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c76aa6bb-c391-4889-9610-1fdcf4f1128b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '182b1efb715147b095fce63a2d9b0ea6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:21.747632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c76aa6bb-c391-4889-9610-1fdcf4f1128b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '182b1efb715147b095fce63a2d9b0ea6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:21.748355 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c76aa6bb-c391-4889-9610-1fdcf4f1128b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '182b1efb715147b095fce63a2d9b0ea6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:21.754932 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1290/2559] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:22:21 2022] DELETE /identity/v3/users/182b1efb715147b095fce63a2d9b0ea6 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:21.760052 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c911c-6982-4fcd-b7c8-a63b813d3288 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:21.760585 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c911c-6982-4fcd-b7c8-a63b813d3288 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:21.760585 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c911c-6982-4fcd-b7c8-a63b813d3288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:21.780781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-884c911c-6982-4fcd-b7c8-a63b813d3288 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:21.781916 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-884c911c-6982-4fcd-b7c8-a63b813d3288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:21.872686 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1270/2560] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:21.886917 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76aa6bb-c391-4889-9610-1fdcf4f1128b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:21.888624 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c76aa6bb-c391-4889-9610-1fdcf4f1128b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:21.963982 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76aa6bb-c391-4889-9610-1fdcf4f1128b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:21.968627 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e05055-77cd-45c3-b6e3-60e3a5b83123 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:21.977400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e05055-77cd-45c3-b6e3-60e3a5b83123 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:21.977400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e05055-77cd-45c3-b6e3-60e3a5b83123 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:21.977400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e05055-77cd-45c3-b6e3-60e3a5b83123 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:21.980580 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e05055-77cd-45c3-b6e3-60e3a5b83123 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:21.982083 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e05055-77cd-45c3-b6e3-60e3a5b83123 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:21.993086 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1291/2561] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:22:22.503525 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c911c-6982-4fcd-b7c8-a63b813d3288 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:22.504998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-884c911c-6982-4fcd-b7c8-a63b813d3288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:22.519226 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c911c-6982-4fcd-b7c8-a63b813d3288 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:22.527332 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff138f-586a-4974-af3b-cf0408f4801a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:22.527332 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff138f-586a-4974-af3b-cf0408f4801a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:22.527332 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff138f-586a-4974-af3b-cf0408f4801a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:22.527332 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff138f-586a-4974-af3b-cf0408f4801a admin admin] PATH_INFO: `/v3/projects/c8b2c5fa7a5847908708c70ae3352fed` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:22.527751 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff138f-586a-4974-af3b-cf0408f4801a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8b2c5fa7a5847908708c70ae3352fed)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:22.528481 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff138f-586a-4974-af3b-cf0408f4801a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:22.661259 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d7ff138f-586a-4974-af3b-cf0408f4801a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8b2c5fa7a5847908708c70ae3352fed', 'request_id': 'req-d7ff138f-586a-4974-af3b-cf0408f4801a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:22.740809 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d7ff138f-586a-4974-af3b-cf0408f4801a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8b2c5fa7a5847908708c70ae3352fed', 'request_id': 'req-d7ff138f-586a-4974-af3b-cf0408f4801a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:22.747844 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d7ff138f-586a-4974-af3b-cf0408f4801a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8b2c5fa7a5847908708c70ae3352fed', 'request_id': 'req-d7ff138f-586a-4974-af3b-cf0408f4801a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:22.760903 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d7ff138f-586a-4974-af3b-cf0408f4801a admin admin] The token cache is being invalidate because project c8b2c5fa7a5847908708c70ae3352fed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:22:22.761576 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d7ff138f-586a-4974-af3b-cf0408f4801a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:22.761881 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d7ff138f-586a-4974-af3b-cf0408f4801a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:22.763453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1271/2562] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:22:22 2022] DELETE /identity/v3/projects/c8b2c5fa7a5847908708c70ae3352fed => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:22.768118 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e05055-77cd-45c3-b6e3-60e3a5b83123 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:22.778019 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e05055-77cd-45c3-b6e3-60e3a5b83123 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:22.836637 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08449e1-d63b-4f4e-a0eb-46c1baf0b05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:22.841020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08449e1-d63b-4f4e-a0eb-46c1baf0b05f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:22.841020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08449e1-d63b-4f4e-a0eb-46c1baf0b05f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:22.841020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08449e1-d63b-4f4e-a0eb-46c1baf0b05f admin admin] PATH_INFO: `/v3/users/9bba4c4beeeb44a5ab5cff7456b04c72` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:22.841020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08449e1-d63b-4f4e-a0eb-46c1baf0b05f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bba4c4beeeb44a5ab5cff7456b04c72)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:22.841020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08449e1-d63b-4f4e-a0eb-46c1baf0b05f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:23.019814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:23.028186 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f08449e1-d63b-4f4e-a0eb-46c1baf0b05f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bba4c4beeeb44a5ab5cff7456b04c72'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:23.140945 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f08449e1-d63b-4f4e-a0eb-46c1baf0b05f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bba4c4beeeb44a5ab5cff7456b04c72'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:23.148613 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f08449e1-d63b-4f4e-a0eb-46c1baf0b05f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bba4c4beeeb44a5ab5cff7456b04c72'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:23.162640 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f08449e1-d63b-4f4e-a0eb-46c1baf0b05f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bba4c4beeeb44a5ab5cff7456b04c72'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:23.163395 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f08449e1-d63b-4f4e-a0eb-46c1baf0b05f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bba4c4beeeb44a5ab5cff7456b04c72'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:23.169901 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1292/2563] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:22:22 2022] DELETE /identity/v3/users/9bba4c4beeeb44a5ab5cff7456b04c72 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:23.640293 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff138f-586a-4974-af3b-cf0408f4801a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:23.641918 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d7ff138f-586a-4974-af3b-cf0408f4801a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:23.661266 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff138f-586a-4974-af3b-cf0408f4801a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:23.705580 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:23.722435 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd4d80-5a35-4d40-a774-8803c6adcf85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:23.723077 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd4d80-5a35-4d40-a774-8803c6adcf85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:23.723420 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd4d80-5a35-4d40-a774-8803c6adcf85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:23.723697 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd4d80-5a35-4d40-a774-8803c6adcf85 admin admin] PATH_INFO: `/v3/projects/6335692f6bbf4abc84cf042d8a40c01f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:23.724838 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd4d80-5a35-4d40-a774-8803c6adcf85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6335692f6bbf4abc84cf042d8a40c01f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:23.726806 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd4d80-5a35-4d40-a774-8803c6adcf85 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:23.890419 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-02fd4d80-5a35-4d40-a774-8803c6adcf85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6335692f6bbf4abc84cf042d8a40c01f', 'request_id': 'req-02fd4d80-5a35-4d40-a774-8803c6adcf85'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:23.969949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-02fd4d80-5a35-4d40-a774-8803c6adcf85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6335692f6bbf4abc84cf042d8a40c01f', 'request_id': 'req-02fd4d80-5a35-4d40-a774-8803c6adcf85'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:23.976834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-02fd4d80-5a35-4d40-a774-8803c6adcf85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6335692f6bbf4abc84cf042d8a40c01f', 'request_id': 'req-02fd4d80-5a35-4d40-a774-8803c6adcf85'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:23.987344 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-02fd4d80-5a35-4d40-a774-8803c6adcf85 admin admin] The token cache is being invalidate because project 6335692f6bbf4abc84cf042d8a40c01f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:22:23.987958 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-02fd4d80-5a35-4d40-a774-8803c6adcf85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:23.988614 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-02fd4d80-5a35-4d40-a774-8803c6adcf85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:23.990216 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1272/2564] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:22:23 2022] DELETE /identity/v3/projects/6335692f6bbf4abc84cf042d8a40c01f => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:23.994272 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08449e1-d63b-4f4e-a0eb-46c1baf0b05f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:23.995738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f08449e1-d63b-4f4e-a0eb-46c1baf0b05f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:24.009033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08449e1-d63b-4f4e-a0eb-46c1baf0b05f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:24.072532 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cfc476-99a8-4d43-b229-5b86f82d10aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:24.077094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cfc476-99a8-4d43-b229-5b86f82d10aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:24.077094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cfc476-99a8-4d43-b229-5b86f82d10aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:24.077094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cfc476-99a8-4d43-b229-5b86f82d10aa admin admin] PATH_INFO: `/v3/users/1d467af2347b4817ba70bcc454f2eee7` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:24.077094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cfc476-99a8-4d43-b229-5b86f82d10aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d467af2347b4817ba70bcc454f2eee7)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:24.077485 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cfc476-99a8-4d43-b229-5b86f82d10aa admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:24.275203 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-05cfc476-99a8-4d43-b229-5b86f82d10aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d467af2347b4817ba70bcc454f2eee7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:24.352472 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-05cfc476-99a8-4d43-b229-5b86f82d10aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d467af2347b4817ba70bcc454f2eee7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:24.360560 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-05cfc476-99a8-4d43-b229-5b86f82d10aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d467af2347b4817ba70bcc454f2eee7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:24.372941 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-05cfc476-99a8-4d43-b229-5b86f82d10aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d467af2347b4817ba70bcc454f2eee7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:24.374284 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-05cfc476-99a8-4d43-b229-5b86f82d10aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d467af2347b4817ba70bcc454f2eee7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:24.380143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1293/2565] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:22:23 2022] DELETE /identity/v3/users/1d467af2347b4817ba70bcc454f2eee7 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:24.839632 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd4d80-5a35-4d40-a774-8803c6adcf85 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:24.841077 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-02fd4d80-5a35-4d40-a774-8803c6adcf85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:24.855997 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd4d80-5a35-4d40-a774-8803c6adcf85 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:24.915633 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2d693-95b8-4114-b56f-72c757d2c037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:24.916358 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2d693-95b8-4114-b56f-72c757d2c037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:24.916590 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2d693-95b8-4114-b56f-72c757d2c037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:24.916866 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2d693-95b8-4114-b56f-72c757d2c037 admin admin] PATH_INFO: `/v3/projects/282f4a5318164e2e938c8988b69cdd50` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:24.917980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2d693-95b8-4114-b56f-72c757d2c037 admin admin] RBAC: Authorizing `identity:delete_project(project_id=282f4a5318164e2e938c8988b69cdd50)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:24.919915 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2d693-95b8-4114-b56f-72c757d2c037 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:25.089471 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6db2d693-95b8-4114-b56f-72c757d2c037 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '282f4a5318164e2e938c8988b69cdd50', 'request_id': 'req-6db2d693-95b8-4114-b56f-72c757d2c037'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:25.261041 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6db2d693-95b8-4114-b56f-72c757d2c037 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '282f4a5318164e2e938c8988b69cdd50', 'request_id': 'req-6db2d693-95b8-4114-b56f-72c757d2c037'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:25.269156 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6db2d693-95b8-4114-b56f-72c757d2c037 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '282f4a5318164e2e938c8988b69cdd50', 'request_id': 'req-6db2d693-95b8-4114-b56f-72c757d2c037'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:25.280333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6db2d693-95b8-4114-b56f-72c757d2c037 admin admin] The token cache is being invalidate because project 282f4a5318164e2e938c8988b69cdd50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:22:25.280647 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6db2d693-95b8-4114-b56f-72c757d2c037 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:25.281426 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6db2d693-95b8-4114-b56f-72c757d2c037 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:25.287265 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1273/2566] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:22:24 2022] DELETE /identity/v3/projects/282f4a5318164e2e938c8988b69cdd50 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:25.288840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2ed71-3025-4a1b-8e42-8952852e56c5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:25.292544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2ed71-3025-4a1b-8e42-8952852e56c5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:25.292544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2ed71-3025-4a1b-8e42-8952852e56c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:25.308893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f6d2ed71-3025-4a1b-8e42-8952852e56c5 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:25.310094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f6d2ed71-3025-4a1b-8e42-8952852e56c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:25.339796 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:25.379913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1294/2567] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:25.385695 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7eb2e-5aaa-442e-8fd2-d3b98a3301c3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:25.386008 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7eb2e-5aaa-442e-8fd2-d3b98a3301c3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:25.386282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7eb2e-5aaa-442e-8fd2-d3b98a3301c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:25.395592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:25.413171 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5bd7eb2e-5aaa-442e-8fd2-d3b98a3301c3 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:25.414665 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5bd7eb2e-5aaa-442e-8fd2-d3b98a3301c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:25.423854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1274/2568] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:25.428583 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2ed71-3025-4a1b-8e42-8952852e56c5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:25.439465 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2ed71-3025-4a1b-8e42-8952852e56c5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:25.447058 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57076849-d83c-4168-8de8-32d5baa87da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:25.447058 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57076849-d83c-4168-8de8-32d5baa87da8 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:25.447058 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57076849-d83c-4168-8de8-32d5baa87da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:25.447058 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57076849-d83c-4168-8de8-32d5baa87da8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:25.448939 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57076849-d83c-4168-8de8-32d5baa87da8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:22:25.448939 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57076849-d83c-4168-8de8-32d5baa87da8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:25.448939 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57076849-d83c-4168-8de8-32d5baa87da8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:25.462517 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1295/2569] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:22:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:22:25.465794 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd7eb2e-5aaa-442e-8fd2-d3b98a3301c3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:25.477058 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd7eb2e-5aaa-442e-8fd2-d3b98a3301c3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:25.487054 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43177f-18f7-43e5-b3e0-426377320278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:25.487054 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43177f-18f7-43e5-b3e0-426377320278 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:25.487054 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43177f-18f7-43e5-b3e0-426377320278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:25.487054 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43177f-18f7-43e5-b3e0-426377320278 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:25.487550 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43177f-18f7-43e5-b3e0-426377320278 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:25.488710 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43177f-18f7-43e5-b3e0-426377320278 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:25.622705 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1275/2570] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:22:25 2022] POST /identity/v3/projects => generated 379 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:22:25.627011 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57076849-d83c-4168-8de8-32d5baa87da8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:25.637299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57076849-d83c-4168-8de8-32d5baa87da8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:25.694375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71a6d3-0490-4a8a-859c-702a00ac578b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:25.695012 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71a6d3-0490-4a8a-859c-702a00ac578b admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:25.695307 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71a6d3-0490-4a8a-859c-702a00ac578b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:25.695576 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71a6d3-0490-4a8a-859c-702a00ac578b admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:25.696188 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa71a6d3-0490-4a8a-859c-702a00ac578b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:25.698101 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa71a6d3-0490-4a8a-859c-702a00ac578b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:25.816094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1296/2571] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:22:25 2022] POST /identity/v3/users => generated 290 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:22:25.820380 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43177f-18f7-43e5-b3e0-426377320278 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:25.832773 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43177f-18f7-43e5-b3e0-426377320278 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:25.837697 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896daea-e4db-40e6-b347-a409cf970a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:25.838290 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896daea-e4db-40e6-b347-a409cf970a54 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:25.838598 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896daea-e4db-40e6-b347-a409cf970a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:25.838864 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896daea-e4db-40e6-b347-a409cf970a54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:25.839533 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896daea-e4db-40e6-b347-a409cf970a54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:25.841100 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896daea-e4db-40e6-b347-a409cf970a54 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:25.851625 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1276/2572] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:25.856194 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71a6d3-0490-4a8a-859c-702a00ac578b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:25.866967 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71a6d3-0490-4a8a-859c-702a00ac578b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:25.871614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206bb4e1-1831-4d6c-aecf-07d49c2f11a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:25.872197 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bb4e1-1831-4d6c-aecf-07d49c2f11a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:25.872470 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bb4e1-1831-4d6c-aecf-07d49c2f11a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:25.872733 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bb4e1-1831-4d6c-aecf-07d49c2f11a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:25.873390 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206bb4e1-1831-4d6c-aecf-07d49c2f11a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:25.875531 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206bb4e1-1831-4d6c-aecf-07d49c2f11a5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:25.886831 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1297/2573] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:25.892394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896daea-e4db-40e6-b347-a409cf970a54 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:25.903056 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896daea-e4db-40e6-b347-a409cf970a54 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:25.907820 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8988c8-ec46-4cfe-860e-dbe267e82b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:25.908456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8988c8-ec46-4cfe-860e-dbe267e82b3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:25.908727 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8988c8-ec46-4cfe-860e-dbe267e82b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:25.908991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8988c8-ec46-4cfe-860e-dbe267e82b3d admin admin] PATH_INFO: `/v3/projects/cab97e8ead7e4f039601c1fa48b78a27/users/3881a47248b74bdc955a075fce3e41cd/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:26.050497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8988c8-ec46-4cfe-860e-dbe267e82b3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cab97e8ead7e4f039601c1fa48b78a27, user_id=3881a47248b74bdc955a075fce3e41cd, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:26.053170 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8988c8-ec46-4cfe-860e-dbe267e82b3d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:26.158099 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1277/2574] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:25 2022] PUT /identity/v3/projects/cab97e8ead7e4f039601c1fa48b78a27/users/3881a47248b74bdc955a075fce3e41cd/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:26.169444 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96281072-5f17-4ace-b6e5-712362cfdf8d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:26.169735 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96281072-5f17-4ace-b6e5-712362cfdf8d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:26.170013 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96281072-5f17-4ace-b6e5-712362cfdf8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:26.198307 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-96281072-5f17-4ace-b6e5-712362cfdf8d None None] MFA Rules not processed for user `3881a47248b74bdc955a075fce3e41cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:26.199543 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-96281072-5f17-4ace-b6e5-712362cfdf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:26.266413 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1298/2575] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:26.271025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8988c8-ec46-4cfe-860e-dbe267e82b3d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:26.330991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8988c8-ec46-4cfe-860e-dbe267e82b3d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:26.335549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00f429-22ae-4ce1-950d-007512427ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:26.336137 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00f429-22ae-4ce1-950d-007512427ea4 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:26.336408 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00f429-22ae-4ce1-950d-007512427ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:26.336672 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00f429-22ae-4ce1-950d-007512427ea4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:26.337305 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c00f429-22ae-4ce1-950d-007512427ea4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:26.339203 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c00f429-22ae-4ce1-950d-007512427ea4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:26.450601 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1278/2576] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:22:26 2022] POST /identity/v3/projects => generated 377 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:22:26.454988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96281072-5f17-4ace-b6e5-712362cfdf8d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:26.466097 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96281072-5f17-4ace-b6e5-712362cfdf8d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:26.493565 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:26.523841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0f83b-c90a-4da4-a841-fd3921ddb6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:26.524447 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0f83b-c90a-4da4-a841-fd3921ddb6d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:26.524727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0f83b-c90a-4da4-a841-fd3921ddb6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:26.524997 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0f83b-c90a-4da4-a841-fd3921ddb6d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:26.525609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e0f83b-c90a-4da4-a841-fd3921ddb6d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:26.527560 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e0f83b-c90a-4da4-a841-fd3921ddb6d1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:26.664818 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1299/2577] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:22:26 2022] POST /identity/v3/users => generated 289 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:22:26.666059 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00f429-22ae-4ce1-950d-007512427ea4 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:26.677502 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00f429-22ae-4ce1-950d-007512427ea4 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:26.682962 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac73690-5473-48f7-96d7-d39b573141da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:26.683636 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac73690-5473-48f7-96d7-d39b573141da admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:26.683935 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac73690-5473-48f7-96d7-d39b573141da admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:26.684303 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac73690-5473-48f7-96d7-d39b573141da admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:26.684839 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac73690-5473-48f7-96d7-d39b573141da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:26.686497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac73690-5473-48f7-96d7-d39b573141da admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:26.697335 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1279/2578] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:26.701686 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0f83b-c90a-4da4-a841-fd3921ddb6d1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:26.712415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0f83b-c90a-4da4-a841-fd3921ddb6d1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:26.717236 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47823989-0af1-471c-8baf-d265d0c77bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:26.717810 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47823989-0af1-471c-8baf-d265d0c77bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:26.718090 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47823989-0af1-471c-8baf-d265d0c77bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:26.718377 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47823989-0af1-471c-8baf-d265d0c77bc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:26.719290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47823989-0af1-471c-8baf-d265d0c77bc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:26.720898 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47823989-0af1-471c-8baf-d265d0c77bc7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:26.731101 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1300/2579] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:22:26.736202 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac73690-5473-48f7-96d7-d39b573141da admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:26.746060 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac73690-5473-48f7-96d7-d39b573141da admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:26.751639 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c58ca-6187-42ea-acfd-316de5abf6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:26.752267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c58ca-6187-42ea-acfd-316de5abf6dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:26.752538 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c58ca-6187-42ea-acfd-316de5abf6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:26.752803 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c58ca-6187-42ea-acfd-316de5abf6dd admin admin] PATH_INFO: `/v3/projects/7e5f9929f2c04adcacd5dcb208127ebc/users/508d0be524114a3fb13ff8c9f18e0b1a/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:26.769594 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c58ca-6187-42ea-acfd-316de5abf6dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e5f9929f2c04adcacd5dcb208127ebc, user_id=508d0be524114a3fb13ff8c9f18e0b1a, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:26.772627 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c58ca-6187-42ea-acfd-316de5abf6dd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:26.863418 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47823989-0af1-471c-8baf-d265d0c77bc7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:26.865220 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-47823989-0af1-471c-8baf-d265d0c77bc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:26.874116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1280/2580] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:26 2022] PUT /identity/v3/projects/7e5f9929f2c04adcacd5dcb208127ebc/users/508d0be524114a3fb13ff8c9f18e0b1a/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:26.891260 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3a829-6b9c-4122-baad-4dbfcc27a24d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:26.891563 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3a829-6b9c-4122-baad-4dbfcc27a24d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:26.891816 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3a829-6b9c-4122-baad-4dbfcc27a24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:26.919562 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-23e3a829-6b9c-4122-baad-4dbfcc27a24d None None] MFA Rules not processed for user `508d0be524114a3fb13ff8c9f18e0b1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:26.920789 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-23e3a829-6b9c-4122-baad-4dbfcc27a24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:26.962372 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47823989-0af1-471c-8baf-d265d0c77bc7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:26.963020 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:26.993581 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1281/2581] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:22:26.998292 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3a829-6b9c-4122-baad-4dbfcc27a24d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:27.018463 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8dac8b-024d-4f7e-bd57-9ce29f1071be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:27.021536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dac8b-024d-4f7e-bd57-9ce29f1071be service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:27.021536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dac8b-024d-4f7e-bd57-9ce29f1071be service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:27.021536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dac8b-024d-4f7e-bd57-9ce29f1071be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:27.021536 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fa8dac8b-024d-4f7e-bd57-9ce29f1071be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:27.063144 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3a829-6b9c-4122-baad-4dbfcc27a24d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:27.067848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00ce02-2a64-41e4-a153-10f9e49cdff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:27.068454 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00ce02-2a64-41e4-a153-10f9e49cdff0 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:27.068728 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00ce02-2a64-41e4-a153-10f9e49cdff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:27.069006 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00ce02-2a64-41e4-a153-10f9e49cdff0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:27.069626 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a00ce02-2a64-41e4-a153-10f9e49cdff0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:27.071561 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a00ce02-2a64-41e4-a153-10f9e49cdff0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:27.083302 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:27.087049 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8dac8b-024d-4f7e-bd57-9ce29f1071be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:27.088509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8dac8b-024d-4f7e-bd57-9ce29f1071be service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:27.098518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1301/2582] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:22:27.208412 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1282/2583] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:22:26 2022] POST /identity/v3/projects => generated 379 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:22:27.209579 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8dac8b-024d-4f7e-bd57-9ce29f1071be service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:27.219379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8dac8b-024d-4f7e-bd57-9ce29f1071be service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:27.302626 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a1cd3-189f-42b4-ad53-757a5e01de8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:27.303407 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a1cd3-189f-42b4-ad53-757a5e01de8c admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:27.303682 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a1cd3-189f-42b4-ad53-757a5e01de8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:27.303950 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a1cd3-189f-42b4-ad53-757a5e01de8c admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:27.305020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a1cd3-189f-42b4-ad53-757a5e01de8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:27.307358 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a1cd3-189f-42b4-ad53-757a5e01de8c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:27.428301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1302/2584] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:22:27 2022] POST /identity/v3/users => generated 296 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:22:27.433117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00ce02-2a64-41e4-a153-10f9e49cdff0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:27.445119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00ce02-2a64-41e4-a153-10f9e49cdff0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:27.449776 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d3d8a-7de1-42e6-b137-534e3862a617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:27.450394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d3d8a-7de1-42e6-b137-534e3862a617 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:27.450674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d3d8a-7de1-42e6-b137-534e3862a617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:27.450943 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d3d8a-7de1-42e6-b137-534e3862a617 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:27.451580 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d3d8a-7de1-42e6-b137-534e3862a617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:27.453366 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d3d8a-7de1-42e6-b137-534e3862a617 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:27.464401 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1283/2585] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:27.469167 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a1cd3-189f-42b4-ad53-757a5e01de8c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:27.480520 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a1cd3-189f-42b4-ad53-757a5e01de8c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:27.494781 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19679bc1-f2bf-411e-be30-06603eaccc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:27.495507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19679bc1-f2bf-411e-be30-06603eaccc69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:27.495783 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19679bc1-f2bf-411e-be30-06603eaccc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:27.496050 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19679bc1-f2bf-411e-be30-06603eaccc69 admin admin] PATH_INFO: `/v3/projects/e62045f65fa04e63aa1ac808625090c3/users/b22ff26d4cb54266ac71b95a21a7234f/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:27.514367 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19679bc1-f2bf-411e-be30-06603eaccc69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e62045f65fa04e63aa1ac808625090c3, user_id=b22ff26d4cb54266ac71b95a21a7234f, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:27.517077 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19679bc1-f2bf-411e-be30-06603eaccc69 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:27.618959 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1303/2586] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:27 2022] PUT /identity/v3/projects/e62045f65fa04e63aa1ac808625090c3/users/b22ff26d4cb54266ac71b95a21a7234f/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:27.622693 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d3d8a-7de1-42e6-b137-534e3862a617 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:27.634432 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d3d8a-7de1-42e6-b137-534e3862a617 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:27.694003 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f50c0f-c2c2-4df5-a94f-7788ebda98a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:27.694604 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f50c0f-c2c2-4df5-a94f-7788ebda98a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:27.694879 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f50c0f-c2c2-4df5-a94f-7788ebda98a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:27.695284 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f50c0f-c2c2-4df5-a94f-7788ebda98a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:27.696026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f50c0f-c2c2-4df5-a94f-7788ebda98a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:27.697617 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f50c0f-c2c2-4df5-a94f-7788ebda98a7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:27.709315 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1284/2587] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:27.713267 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19679bc1-f2bf-411e-be30-06603eaccc69 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:27.724497 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19679bc1-f2bf-411e-be30-06603eaccc69 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:27.729181 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88176c6-0017-44d4-91d5-04843e4ea7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:27.729840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88176c6-0017-44d4-91d5-04843e4ea7ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:27.730115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88176c6-0017-44d4-91d5-04843e4ea7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:27.730418 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88176c6-0017-44d4-91d5-04843e4ea7ba admin admin] PATH_INFO: `/v3/projects/e62045f65fa04e63aa1ac808625090c3/users/b22ff26d4cb54266ac71b95a21a7234f/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:27.732310 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88176c6-0017-44d4-91d5-04843e4ea7ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=e62045f65fa04e63aa1ac808625090c3, user_id=b22ff26d4cb54266ac71b95a21a7234f, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:27.735038 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88176c6-0017-44d4-91d5-04843e4ea7ba admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:27.745706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1304/2588] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:27 2022] PUT /identity/v3/projects/e62045f65fa04e63aa1ac808625090c3/users/b22ff26d4cb54266ac71b95a21a7234f/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:27.757901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516f877-3524-4f03-9956-a8f194586516 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:27.757901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516f877-3524-4f03-9956-a8f194586516 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:27.764105 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516f877-3524-4f03-9956-a8f194586516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:27.791867 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f516f877-3524-4f03-9956-a8f194586516 None None] MFA Rules not processed for user `b22ff26d4cb54266ac71b95a21a7234f`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:27.793083 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f516f877-3524-4f03-9956-a8f194586516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:27.835356 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:27.863621 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1285/2589] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:27 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:27.875737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88176c6-0017-44d4-91d5-04843e4ea7ba admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:27.877947 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e88176c6-0017-44d4-91d5-04843e4ea7ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:27.939017 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88176c6-0017-44d4-91d5-04843e4ea7ba admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:27.942850 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce7ed1-1a01-4031-bad7-1b028e09c2bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:27.943491 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce7ed1-1a01-4031-bad7-1b028e09c2bd service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:27.943781 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce7ed1-1a01-4031-bad7-1b028e09c2bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:27.944045 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce7ed1-1a01-4031-bad7-1b028e09c2bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:27.959213 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce7ed1-1a01-4031-bad7-1b028e09c2bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:27.967548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce7ed1-1a01-4031-bad7-1b028e09c2bd service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:27.971106 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1305/2590] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:27 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:22:28.096504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516f877-3524-4f03-9956-a8f194586516 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:28.097935 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f516f877-3524-4f03-9956-a8f194586516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:28.114183 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516f877-3524-4f03-9956-a8f194586516 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:28.117704 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65fc597-b3a1-4521-bb3d-0b0bd34c2332 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:28.118443 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fc597-b3a1-4521-bb3d-0b0bd34c2332 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:28.118696 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fc597-b3a1-4521-bb3d-0b0bd34c2332 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:28.119050 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fc597-b3a1-4521-bb3d-0b0bd34c2332 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:28.188647 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65fc597-b3a1-4521-bb3d-0b0bd34c2332 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:28.190646 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65fc597-b3a1-4521-bb3d-0b0bd34c2332 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:28.200144 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1286/2591] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:22:30.942230 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce7ed1-1a01-4031-bad7-1b028e09c2bd service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:30.943733 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9dce7ed1-1a01-4031-bad7-1b028e09c2bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:31.000294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:31.018435 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce7ed1-1a01-4031-bad7-1b028e09c2bd service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:31.022993 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d9973-666c-407e-abdc-baadb8fc2803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:31.023692 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d9973-666c-407e-abdc-baadb8fc2803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:31.023942 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d9973-666c-407e-abdc-baadb8fc2803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:31.024215 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d9973-666c-407e-abdc-baadb8fc2803 admin admin] PATH_INFO: `/v3/users/5bf9229cd5d6429ea14249f8976e83cc` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:31.025479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d9973-666c-407e-abdc-baadb8fc2803 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bf9229cd5d6429ea14249f8976e83cc)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:31.027450 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d9973-666c-407e-abdc-baadb8fc2803 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:31.231057 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c57d9973-666c-407e-abdc-baadb8fc2803 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bf9229cd5d6429ea14249f8976e83cc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:31.306367 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c57d9973-666c-407e-abdc-baadb8fc2803 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bf9229cd5d6429ea14249f8976e83cc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:31.313848 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c57d9973-666c-407e-abdc-baadb8fc2803 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bf9229cd5d6429ea14249f8976e83cc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:31.326109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c57d9973-666c-407e-abdc-baadb8fc2803 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bf9229cd5d6429ea14249f8976e83cc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:31.326795 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c57d9973-666c-407e-abdc-baadb8fc2803 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bf9229cd5d6429ea14249f8976e83cc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:31.333553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1306/2592] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:22:30 2022] DELETE /identity/v3/users/5bf9229cd5d6429ea14249f8976e83cc => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:31.577099 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65fc597-b3a1-4521-bb3d-0b0bd34c2332 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:31.589597 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65fc597-b3a1-4521-bb3d-0b0bd34c2332 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:31.653514 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c0a6e-cd7b-44d1-a34d-76d671190234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:31.657975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c0a6e-cd7b-44d1-a34d-76d671190234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:31.657975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c0a6e-cd7b-44d1-a34d-76d671190234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:31.657975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c0a6e-cd7b-44d1-a34d-76d671190234 admin admin] PATH_INFO: `/v3/projects/ed45fd312c58480f993616d451abcab7` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:31.657975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c0a6e-cd7b-44d1-a34d-76d671190234 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed45fd312c58480f993616d451abcab7)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:31.657975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c0a6e-cd7b-44d1-a34d-76d671190234 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:31.759174 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-aa5c0a6e-cd7b-44d1-a34d-76d671190234 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed45fd312c58480f993616d451abcab7', 'request_id': 'req-aa5c0a6e-cd7b-44d1-a34d-76d671190234'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:31.895040 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-aa5c0a6e-cd7b-44d1-a34d-76d671190234 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed45fd312c58480f993616d451abcab7', 'request_id': 'req-aa5c0a6e-cd7b-44d1-a34d-76d671190234'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:31.902055 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-aa5c0a6e-cd7b-44d1-a34d-76d671190234 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed45fd312c58480f993616d451abcab7', 'request_id': 'req-aa5c0a6e-cd7b-44d1-a34d-76d671190234'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:31.913635 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-aa5c0a6e-cd7b-44d1-a34d-76d671190234 admin admin] The token cache is being invalidate because project ed45fd312c58480f993616d451abcab7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:22:31.914083 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-aa5c0a6e-cd7b-44d1-a34d-76d671190234 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:31.914825 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-aa5c0a6e-cd7b-44d1-a34d-76d671190234 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:31.916678 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1287/2593] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:22:31 2022] DELETE /identity/v3/projects/ed45fd312c58480f993616d451abcab7 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:31.921716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244ab7f-fe5c-4373-876d-cdfe13cb0b59 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:31.922029 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244ab7f-fe5c-4373-876d-cdfe13cb0b59 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:31.928241 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244ab7f-fe5c-4373-876d-cdfe13cb0b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:31.943037 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9244ab7f-fe5c-4373-876d-cdfe13cb0b59 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:31.944278 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9244ab7f-fe5c-4373-876d-cdfe13cb0b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:32.002971 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1307/2594] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:32.011034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbc5ad-121d-472b-b6f3-ce000ee235c6 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:32.011317 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbc5ad-121d-472b-b6f3-ce000ee235c6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:32.011589 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbc5ad-121d-472b-b6f3-ce000ee235c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:32.021101 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:32.032007 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-72cbc5ad-121d-472b-b6f3-ce000ee235c6 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:32.033356 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-72cbc5ad-121d-472b-b6f3-ce000ee235c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:32.041836 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1288/2595] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:22:32.046074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244ab7f-fe5c-4373-876d-cdfe13cb0b59 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:32.055712 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244ab7f-fe5c-4373-876d-cdfe13cb0b59 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:32.060236 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236dd5cc-4f5c-47b1-8c35-ae14a03c177a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:32.060823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236dd5cc-4f5c-47b1-8c35-ae14a03c177a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:32.061095 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236dd5cc-4f5c-47b1-8c35-ae14a03c177a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:32.061363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236dd5cc-4f5c-47b1-8c35-ae14a03c177a admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:32.062000 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236dd5cc-4f5c-47b1-8c35-ae14a03c177a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:22:32.062301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236dd5cc-4f5c-47b1-8c35-ae14a03c177a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:32.063970 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236dd5cc-4f5c-47b1-8c35-ae14a03c177a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:32.075379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1308/2596] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:22:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:22:32.079587 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbc5ad-121d-472b-b6f3-ce000ee235c6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:32.090031 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbc5ad-121d-472b-b6f3-ce000ee235c6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:32.094956 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a996a5-543e-4b7d-860b-cb168ce23502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:32.095698 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a996a5-543e-4b7d-860b-cb168ce23502 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:32.096472 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a996a5-543e-4b7d-860b-cb168ce23502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:32.096745 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a996a5-543e-4b7d-860b-cb168ce23502 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:32.097380 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a996a5-543e-4b7d-860b-cb168ce23502 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:32.099380 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a996a5-543e-4b7d-860b-cb168ce23502 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:32.176579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1289/2597] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:22:32 2022] POST /identity/v3/projects => generated 403 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:22:32.180986 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236dd5cc-4f5c-47b1-8c35-ae14a03c177a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:32.191558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236dd5cc-4f5c-47b1-8c35-ae14a03c177a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:32.242413 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:32.249073 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d018d-2ed2-473a-a8ca-81132941c3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:32.249677 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d018d-2ed2-473a-a8ca-81132941c3f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:32.250328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d018d-2ed2-473a-a8ca-81132941c3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:32.250328 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d018d-2ed2-473a-a8ca-81132941c3f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:32.250913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d018d-2ed2-473a-a8ca-81132941c3f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:32.253743 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d018d-2ed2-473a-a8ca-81132941c3f8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:32.404428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1309/2598] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:22:32 2022] POST /identity/v3/users => generated 302 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:22:32.409158 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a996a5-543e-4b7d-860b-cb168ce23502 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:32.419418 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a996a5-543e-4b7d-860b-cb168ce23502 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:32.424680 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0a08d-0a35-4e38-859c-4f53c9fb3e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:32.425452 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0a08d-0a35-4e38-859c-4f53c9fb3e6e admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:32.425735 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0a08d-0a35-4e38-859c-4f53c9fb3e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:32.426001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0a08d-0a35-4e38-859c-4f53c9fb3e6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:32.426736 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a0a08d-0a35-4e38-859c-4f53c9fb3e6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:32.428278 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a0a08d-0a35-4e38-859c-4f53c9fb3e6e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:32.442272 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1290/2599] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:32.445211 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d018d-2ed2-473a-a8ca-81132941c3f8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:32.456415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d018d-2ed2-473a-a8ca-81132941c3f8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:32.461645 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ffafe1-bcc5-47ff-8f8c-61a6c7070cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:32.462276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffafe1-bcc5-47ff-8f8c-61a6c7070cef admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:32.462594 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffafe1-bcc5-47ff-8f8c-61a6c7070cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:32.462880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffafe1-bcc5-47ff-8f8c-61a6c7070cef admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:32.463517 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ffafe1-bcc5-47ff-8f8c-61a6c7070cef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:32.465126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ffafe1-bcc5-47ff-8f8c-61a6c7070cef admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:32.477396 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1310/2600] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:32.485381 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0a08d-0a35-4e38-859c-4f53c9fb3e6e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:32.497870 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0a08d-0a35-4e38-859c-4f53c9fb3e6e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:32.502521 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4bd5c-4f6b-42d0-b05b-8d8109ef4029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:32.503162 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4bd5c-4f6b-42d0-b05b-8d8109ef4029 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:32.503432 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4bd5c-4f6b-42d0-b05b-8d8109ef4029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:32.503702 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4bd5c-4f6b-42d0-b05b-8d8109ef4029 admin admin] PATH_INFO: `/v3/projects/4f201e9f902342279140cf9a6113845b/users/614d8673327845d6a86fb651557c2aa0/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:32.518303 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b4bd5c-4f6b-42d0-b05b-8d8109ef4029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f201e9f902342279140cf9a6113845b, user_id=614d8673327845d6a86fb651557c2aa0, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:32.520954 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b4bd5c-4f6b-42d0-b05b-8d8109ef4029 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:32.661904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1291/2601] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:32 2022] PUT /identity/v3/projects/4f201e9f902342279140cf9a6113845b/users/614d8673327845d6a86fb651557c2aa0/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:32.667060 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25601699-ead2-4780-9b71-fe75d55fa495 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:32.667415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25601699-ead2-4780-9b71-fe75d55fa495 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:32.667701 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25601699-ead2-4780-9b71-fe75d55fa495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:32.686968 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-25601699-ead2-4780-9b71-fe75d55fa495 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:32.688086 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-25601699-ead2-4780-9b71-fe75d55fa495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:32.745643 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1311/2602] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:32.756176 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4bd5c-4f6b-42d0-b05b-8d8109ef4029 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:32.758200 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e0b4bd5c-4f6b-42d0-b05b-8d8109ef4029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:32.814476 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4bd5c-4f6b-42d0-b05b-8d8109ef4029 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:32.818053 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad08773-341e-48e7-815c-2fadfc5ca46a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:32.818674 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad08773-341e-48e7-815c-2fadfc5ca46a service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:32.819009 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad08773-341e-48e7-815c-2fadfc5ca46a service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:32.819211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad08773-341e-48e7-815c-2fadfc5ca46a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:32.827964 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad08773-341e-48e7-815c-2fadfc5ca46a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:32.829374 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad08773-341e-48e7-815c-2fadfc5ca46a service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:32.838520 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1292/2603] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:22:34.429726 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d66513-df96-476c-911c-061ed6ef9a34 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:34.429726 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d66513-df96-476c-911c-061ed6ef9a34 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:34.429726 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d66513-df96-476c-911c-061ed6ef9a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:34.468693 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-13d66513-df96-476c-911c-061ed6ef9a34 None None] MFA Rules not processed for user `614d8673327845d6a86fb651557c2aa0`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:34.470073 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-13d66513-df96-476c-911c-061ed6ef9a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:34.554152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1312/2604] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:34 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:34.558701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad08773-341e-48e7-815c-2fadfc5ca46a service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:34.577877 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad08773-341e-48e7-815c-2fadfc5ca46a service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:34.582976 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77668709-be71-4921-b359-b8171c0f773f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:34.583703 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77668709-be71-4921-b359-b8171c0f773f admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:34.584034 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77668709-be71-4921-b359-b8171c0f773f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:34.584490 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77668709-be71-4921-b359-b8171c0f773f admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:34.585312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77668709-be71-4921-b359-b8171c0f773f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:34.587547 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77668709-be71-4921-b359-b8171c0f773f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:34.762910 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1293/2605] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:22:34 2022] POST /identity/v3/projects => generated 403 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:22:34.764019 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d66513-df96-476c-911c-061ed6ef9a34 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:34.776588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d66513-df96-476c-911c-061ed6ef9a34 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:34.835446 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c6cd5-8e71-41a5-ba50-d08153fbce81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:34.835974 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c6cd5-8e71-41a5-ba50-d08153fbce81 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:34.836284 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c6cd5-8e71-41a5-ba50-d08153fbce81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:34.836610 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c6cd5-8e71-41a5-ba50-d08153fbce81 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:34.837273 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c6cd5-8e71-41a5-ba50-d08153fbce81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:34.839234 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c6cd5-8e71-41a5-ba50-d08153fbce81 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:34.849190 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:34.944758 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77668709-be71-4921-b359-b8171c0f773f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:34.946767 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-77668709-be71-4921-b359-b8171c0f773f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:34.955931 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1313/2606] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:22:34 2022] POST /identity/v3/users => generated 308 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:22:34.957752 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c6cd5-8e71-41a5-ba50-d08153fbce81 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:34.960749 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77668709-be71-4921-b359-b8171c0f773f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:34.966108 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83356ea7-34d2-4ee6-8b4f-fb8cd256de98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:34.966870 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83356ea7-34d2-4ee6-8b4f-fb8cd256de98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:34.967199 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83356ea7-34d2-4ee6-8b4f-fb8cd256de98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:34.967533 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83356ea7-34d2-4ee6-8b4f-fb8cd256de98 admin admin] PATH_INFO: `/v3/users/3881a47248b74bdc955a075fce3e41cd` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:34.968299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c6cd5-8e71-41a5-ba50-d08153fbce81 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:34.969382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83356ea7-34d2-4ee6-8b4f-fb8cd256de98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3881a47248b74bdc955a075fce3e41cd)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:34.971542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83356ea7-34d2-4ee6-8b4f-fb8cd256de98 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:34.973187 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76b82f-56ff-43f4-b922-fe0342c72d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:34.973778 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76b82f-56ff-43f4-b922-fe0342c72d51 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:34.974055 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76b82f-56ff-43f4-b922-fe0342c72d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:34.974325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76b82f-56ff-43f4-b922-fe0342c72d51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:34.974996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76b82f-56ff-43f4-b922-fe0342c72d51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:34.976578 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76b82f-56ff-43f4-b922-fe0342c72d51 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:34.987700 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1314/2607] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:34.991857 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76b82f-56ff-43f4-b922-fe0342c72d51 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:35.001515 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76b82f-56ff-43f4-b922-fe0342c72d51 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:35.007119 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f127c51-b3a9-407a-86a2-bf27a626b3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:35.007751 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f127c51-b3a9-407a-86a2-bf27a626b3db admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:35.008031 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f127c51-b3a9-407a-86a2-bf27a626b3db admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:35.008308 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f127c51-b3a9-407a-86a2-bf27a626b3db admin admin] PATH_INFO: `/v3/projects/51c96cb7c6c94223b1efbf2767117ca8/users/4d2c3b26061347c8820cde695e1353af/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:35.023334 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f127c51-b3a9-407a-86a2-bf27a626b3db admin admin] RBAC: Authorizing `identity:create_grant(project_id=51c96cb7c6c94223b1efbf2767117ca8, user_id=4d2c3b26061347c8820cde695e1353af, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:35.025966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f127c51-b3a9-407a-86a2-bf27a626b3db admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:35.169909 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1315/2608] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:34 2022] PUT /identity/v3/projects/51c96cb7c6c94223b1efbf2767117ca8/users/4d2c3b26061347c8820cde695e1353af/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:22:35.173363 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f127c51-b3a9-407a-86a2-bf27a626b3db admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:35.223180 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-83356ea7-34d2-4ee6-8b4f-fb8cd256de98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3881a47248b74bdc955a075fce3e41cd'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:35.233529 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f127c51-b3a9-407a-86a2-bf27a626b3db admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:35.238978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a80df-d8a8-4309-ab95-ccb5ae590276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:35.239594 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a80df-d8a8-4309-ab95-ccb5ae590276 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:35.239868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a80df-d8a8-4309-ab95-ccb5ae590276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:35.240132 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a80df-d8a8-4309-ab95-ccb5ae590276 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:35.240790 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a80df-d8a8-4309-ab95-ccb5ae590276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:35.242418 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a80df-d8a8-4309-ab95-ccb5ae590276 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:35.252199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1316/2609] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:22:35.256817 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a80df-d8a8-4309-ab95-ccb5ae590276 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:35.266499 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a80df-d8a8-4309-ab95-ccb5ae590276 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:35.271392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c10ed-5515-4a81-94cf-42d46ce3191d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:35.272021 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c10ed-5515-4a81-94cf-42d46ce3191d admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:35.272290 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c10ed-5515-4a81-94cf-42d46ce3191d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:35.272623 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c10ed-5515-4a81-94cf-42d46ce3191d admin admin] PATH_INFO: `/v3/projects/51c96cb7c6c94223b1efbf2767117ca8/users/4d2c3b26061347c8820cde695e1353af/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:35.274426 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4c10ed-5515-4a81-94cf-42d46ce3191d admin admin] RBAC: Authorizing `identity:create_grant(project_id=51c96cb7c6c94223b1efbf2767117ca8, user_id=4d2c3b26061347c8820cde695e1353af, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:35.277025 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4c10ed-5515-4a81-94cf-42d46ce3191d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:35.287104 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1317/2610] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:35 2022] PUT /identity/v3/projects/51c96cb7c6c94223b1efbf2767117ca8/users/4d2c3b26061347c8820cde695e1353af/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:22:35.346597 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-83356ea7-34d2-4ee6-8b4f-fb8cd256de98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3881a47248b74bdc955a075fce3e41cd'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:35.359470 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-83356ea7-34d2-4ee6-8b4f-fb8cd256de98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3881a47248b74bdc955a075fce3e41cd'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:35.360126 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-83356ea7-34d2-4ee6-8b4f-fb8cd256de98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3881a47248b74bdc955a075fce3e41cd'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:35.360738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-83356ea7-34d2-4ee6-8b4f-fb8cd256de98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3881a47248b74bdc955a075fce3e41cd'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:35.365921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:35.374222 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1294/2611] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:22:34 2022] DELETE /identity/v3/users/3881a47248b74bdc955a075fce3e41cd => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:22:35.379646 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5a3b0-1be5-4750-8f10-88f1583b3374 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:35.379957 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5a3b0-1be5-4750-8f10-88f1583b3374 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:35.380226 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5a3b0-1be5-4750-8f10-88f1583b3374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:35.399780 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e7b5a3b0-1be5-4750-8f10-88f1583b3374 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:35.400980 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e7b5a3b0-1be5-4750-8f10-88f1583b3374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:35.463326 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1318/2612] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:35.475478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83356ea7-34d2-4ee6-8b4f-fb8cd256de98 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:35.477049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-83356ea7-34d2-4ee6-8b4f-fb8cd256de98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:35.538464 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83356ea7-34d2-4ee6-8b4f-fb8cd256de98 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:35.542053 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00e5727-8314-4e28-86a2-3e02708233f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:35.542676 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e5727-8314-4e28-86a2-3e02708233f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:35.542948 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e5727-8314-4e28-86a2-3e02708233f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:35.543211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e5727-8314-4e28-86a2-3e02708233f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:35.557557 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00e5727-8314-4e28-86a2-3e02708233f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:35.557557 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00e5727-8314-4e28-86a2-3e02708233f1 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:35.565439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1295/2613] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:22:36.034202 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5a3b0-1be5-4750-8f10-88f1583b3374 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:36.036770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e7b5a3b0-1be5-4750-8f10-88f1583b3374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:36.048834 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5a3b0-1be5-4750-8f10-88f1583b3374 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:36.054580 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5678f-1a64-4399-b6e7-27dd791b57d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:36.055227 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5678f-1a64-4399-b6e7-27dd791b57d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:36.055507 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5678f-1a64-4399-b6e7-27dd791b57d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:36.055778 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5678f-1a64-4399-b6e7-27dd791b57d7 admin admin] PATH_INFO: `/v3/projects/cab97e8ead7e4f039601c1fa48b78a27` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:36.056927 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5678f-1a64-4399-b6e7-27dd791b57d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cab97e8ead7e4f039601c1fa48b78a27)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:36.058980 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5678f-1a64-4399-b6e7-27dd791b57d7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:36.547668 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-23c5678f-1a64-4399-b6e7-27dd791b57d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cab97e8ead7e4f039601c1fa48b78a27', 'request_id': 'req-23c5678f-1a64-4399-b6e7-27dd791b57d7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:36.554649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-23c5678f-1a64-4399-b6e7-27dd791b57d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cab97e8ead7e4f039601c1fa48b78a27', 'request_id': 'req-23c5678f-1a64-4399-b6e7-27dd791b57d7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:36.633301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-23c5678f-1a64-4399-b6e7-27dd791b57d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cab97e8ead7e4f039601c1fa48b78a27', 'request_id': 'req-23c5678f-1a64-4399-b6e7-27dd791b57d7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:36.636583 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=3,keystone:SELECT=94,keystone:DELETE=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:36.656458 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-23c5678f-1a64-4399-b6e7-27dd791b57d7 admin admin] The token cache is being invalidate because project cab97e8ead7e4f039601c1fa48b78a27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:22:36.656873 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-23c5678f-1a64-4399-b6e7-27dd791b57d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:36.657636 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-23c5678f-1a64-4399-b6e7-27dd791b57d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:36.660413 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1319/2614] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:22:36 2022] DELETE /identity/v3/projects/cab97e8ead7e4f039601c1fa48b78a27 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:36.664317 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00e5727-8314-4e28-86a2-3e02708233f1 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:36.674330 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00e5727-8314-4e28-86a2-3e02708233f1 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:36.730709 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320821e-30a7-4a97-8926-9931137288bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:36.731336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320821e-30a7-4a97-8926-9931137288bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:36.731613 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320821e-30a7-4a97-8926-9931137288bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:36.731880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320821e-30a7-4a97-8926-9931137288bb admin admin] PATH_INFO: `/v3/users/508d0be524114a3fb13ff8c9f18e0b1a` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:36.732986 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1320821e-30a7-4a97-8926-9931137288bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=508d0be524114a3fb13ff8c9f18e0b1a)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:36.734937 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1320821e-30a7-4a97-8926-9931137288bb admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:37.035996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d08eb3-05cc-4e02-ae13-9f921488d3e3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:37.041095 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d08eb3-05cc-4e02-ae13-9f921488d3e3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:37.041532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1320821e-30a7-4a97-8926-9931137288bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '508d0be524114a3fb13ff8c9f18e0b1a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:37.041654 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d08eb3-05cc-4e02-ae13-9f921488d3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:37.064275 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-57d08eb3-05cc-4e02-ae13-9f921488d3e3 None None] MFA Rules not processed for user `4d2c3b26061347c8820cde695e1353af`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:37.065563 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-57d08eb3-05cc-4e02-ae13-9f921488d3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:37.134668 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1320/2615] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:37 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:22:37.163560 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1320821e-30a7-4a97-8926-9931137288bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '508d0be524114a3fb13ff8c9f18e0b1a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:37.176722 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1320821e-30a7-4a97-8926-9931137288bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '508d0be524114a3fb13ff8c9f18e0b1a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:37.177435 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1320821e-30a7-4a97-8926-9931137288bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '508d0be524114a3fb13ff8c9f18e0b1a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:37.178047 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1320821e-30a7-4a97-8926-9931137288bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '508d0be524114a3fb13ff8c9f18e0b1a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:37.190230 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1296/2616] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:22:36 2022] DELETE /identity/v3/users/508d0be524114a3fb13ff8c9f18e0b1a => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:37.788141 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d08eb3-05cc-4e02-ae13-9f921488d3e3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:37.789699 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-57d08eb3-05cc-4e02-ae13-9f921488d3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:37.803678 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d08eb3-05cc-4e02-ae13-9f921488d3e3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:37.861480 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038302c6-834b-4fd9-a726-ae5af568c936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:37.862139 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038302c6-834b-4fd9-a726-ae5af568c936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:37.862442 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038302c6-834b-4fd9-a726-ae5af568c936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:37.862741 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038302c6-834b-4fd9-a726-ae5af568c936 admin admin] PATH_INFO: `/v3/projects/7e5f9929f2c04adcacd5dcb208127ebc` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:37.864029 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038302c6-834b-4fd9-a726-ae5af568c936 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e5f9929f2c04adcacd5dcb208127ebc)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:37.866642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038302c6-834b-4fd9-a726-ae5af568c936 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:37.988030 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-038302c6-834b-4fd9-a726-ae5af568c936 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e5f9929f2c04adcacd5dcb208127ebc', 'request_id': 'req-038302c6-834b-4fd9-a726-ae5af568c936'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:37.995016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-038302c6-834b-4fd9-a726-ae5af568c936 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e5f9929f2c04adcacd5dcb208127ebc', 'request_id': 'req-038302c6-834b-4fd9-a726-ae5af568c936'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:38.095280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-038302c6-834b-4fd9-a726-ae5af568c936 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e5f9929f2c04adcacd5dcb208127ebc', 'request_id': 'req-038302c6-834b-4fd9-a726-ae5af568c936'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:38.106820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-038302c6-834b-4fd9-a726-ae5af568c936 admin admin] The token cache is being invalidate because project 7e5f9929f2c04adcacd5dcb208127ebc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:22:38.107107 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-038302c6-834b-4fd9-a726-ae5af568c936 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:38.107747 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-038302c6-834b-4fd9-a726-ae5af568c936 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:38.109236 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1321/2617] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:22:37 2022] DELETE /identity/v3/projects/7e5f9929f2c04adcacd5dcb208127ebc => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:38.113926 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320821e-30a7-4a97-8926-9931137288bb admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:38.115344 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1320821e-30a7-4a97-8926-9931137288bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:38.128130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320821e-30a7-4a97-8926-9931137288bb admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:38.140509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:38.174983 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:38.186499 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13985fa-ac47-400d-9367-bb0599d22a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:38.187053 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13985fa-ac47-400d-9367-bb0599d22a77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:38.187333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13985fa-ac47-400d-9367-bb0599d22a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:38.187606 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13985fa-ac47-400d-9367-bb0599d22a77 admin admin] PATH_INFO: `/v3/users/b22ff26d4cb54266ac71b95a21a7234f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:38.188766 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13985fa-ac47-400d-9367-bb0599d22a77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b22ff26d4cb54266ac71b95a21a7234f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:38.192149 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13985fa-ac47-400d-9367-bb0599d22a77 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:38.420066 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b13985fa-ac47-400d-9367-bb0599d22a77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b22ff26d4cb54266ac71b95a21a7234f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:38.523755 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b13985fa-ac47-400d-9367-bb0599d22a77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b22ff26d4cb54266ac71b95a21a7234f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:38.537362 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b13985fa-ac47-400d-9367-bb0599d22a77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b22ff26d4cb54266ac71b95a21a7234f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:38.538046 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b13985fa-ac47-400d-9367-bb0599d22a77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b22ff26d4cb54266ac71b95a21a7234f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:38.539895 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b13985fa-ac47-400d-9367-bb0599d22a77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b22ff26d4cb54266ac71b95a21a7234f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:38.551566 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1297/2618] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:22:38 2022] DELETE /identity/v3/users/b22ff26d4cb54266ac71b95a21a7234f => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:38.661046 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038302c6-834b-4fd9-a726-ae5af568c936 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:38.662571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-038302c6-834b-4fd9-a726-ae5af568c936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:38.675982 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038302c6-834b-4fd9-a726-ae5af568c936 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:38.736102 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663c79b-d604-40bb-abfa-4bf2abaf63e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:38.736741 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663c79b-d604-40bb-abfa-4bf2abaf63e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:38.737016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663c79b-d604-40bb-abfa-4bf2abaf63e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:38.737283 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663c79b-d604-40bb-abfa-4bf2abaf63e5 admin admin] PATH_INFO: `/v3/users/614d8673327845d6a86fb651557c2aa0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:38.739246 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7663c79b-d604-40bb-abfa-4bf2abaf63e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=614d8673327845d6a86fb651557c2aa0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:38.741160 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7663c79b-d604-40bb-abfa-4bf2abaf63e5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:38.945152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7663c79b-d604-40bb-abfa-4bf2abaf63e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '614d8673327845d6a86fb651557c2aa0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:39.100850 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7663c79b-d604-40bb-abfa-4bf2abaf63e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '614d8673327845d6a86fb651557c2aa0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:39.108835 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13985fa-ac47-400d-9367-bb0599d22a77 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:39.109282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7663c79b-d604-40bb-abfa-4bf2abaf63e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '614d8673327845d6a86fb651557c2aa0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:39.110450 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b13985fa-ac47-400d-9367-bb0599d22a77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:39.123603 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7663c79b-d604-40bb-abfa-4bf2abaf63e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '614d8673327845d6a86fb651557c2aa0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:39.127244 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7663c79b-d604-40bb-abfa-4bf2abaf63e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '614d8673327845d6a86fb651557c2aa0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:39.127386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13985fa-ac47-400d-9367-bb0599d22a77 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:39.131658 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1322/2619] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:22:38 2022] DELETE /identity/v3/users/614d8673327845d6a86fb651557c2aa0 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:39.132755 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a19b55-b273-4b81-b467-010c3d1d5ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:39.134236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a19b55-b273-4b81-b467-010c3d1d5ef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:39.134562 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a19b55-b273-4b81-b467-010c3d1d5ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:39.135184 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a19b55-b273-4b81-b467-010c3d1d5ef8 admin admin] PATH_INFO: `/v3/projects/e62045f65fa04e63aa1ac808625090c3` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:39.136505 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a19b55-b273-4b81-b467-010c3d1d5ef8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e62045f65fa04e63aa1ac808625090c3)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:39.138339 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a19b55-b273-4b81-b467-010c3d1d5ef8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:39.273955 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c0a19b55-b273-4b81-b467-010c3d1d5ef8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e62045f65fa04e63aa1ac808625090c3', 'request_id': 'req-c0a19b55-b273-4b81-b467-010c3d1d5ef8'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:39.385659 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c0a19b55-b273-4b81-b467-010c3d1d5ef8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e62045f65fa04e63aa1ac808625090c3', 'request_id': 'req-c0a19b55-b273-4b81-b467-010c3d1d5ef8'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:39.392620 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c0a19b55-b273-4b81-b467-010c3d1d5ef8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e62045f65fa04e63aa1ac808625090c3', 'request_id': 'req-c0a19b55-b273-4b81-b467-010c3d1d5ef8'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:39.403998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c0a19b55-b273-4b81-b467-010c3d1d5ef8 admin admin] The token cache is being invalidate because project e62045f65fa04e63aa1ac808625090c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:22:39.404284 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c0a19b55-b273-4b81-b467-010c3d1d5ef8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:39.404937 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c0a19b55-b273-4b81-b467-010c3d1d5ef8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:39.406573 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1298/2620] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:22:39 2022] DELETE /identity/v3/projects/e62045f65fa04e63aa1ac808625090c3 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:22:39.412590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71521a86-6d8e-4b83-82ef-0ccabd001204 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:39.412885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71521a86-6d8e-4b83-82ef-0ccabd001204 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:39.413153 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71521a86-6d8e-4b83-82ef-0ccabd001204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:39.432745 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a19b55-b273-4b81-b467-010c3d1d5ef8 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:39.434773 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c0a19b55-b273-4b81-b467-010c3d1d5ef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:39.438041 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-71521a86-6d8e-4b83-82ef-0ccabd001204 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:39.439514 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-71521a86-6d8e-4b83-82ef-0ccabd001204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:39.497549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:39.512705 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a19b55-b273-4b81-b467-010c3d1d5ef8 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:39.516615 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1323/2621] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:39.520521 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c78863-d135-4325-a1ac-3a5aaa8d25b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:39.521286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c78863-d135-4325-a1ac-3a5aaa8d25b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:39.521903 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c78863-d135-4325-a1ac-3a5aaa8d25b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:39.522115 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c78863-d135-4325-a1ac-3a5aaa8d25b3 admin admin] PATH_INFO: `/v3/projects/4f201e9f902342279140cf9a6113845b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:39.523418 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c78863-d135-4325-a1ac-3a5aaa8d25b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f201e9f902342279140cf9a6113845b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:39.524763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:39.525667 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c78863-d135-4325-a1ac-3a5aaa8d25b3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:39.526884 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97494a7-db55-4ac7-a1b8-218b3cc0dd7f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:39.527490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97494a7-db55-4ac7-a1b8-218b3cc0dd7f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:39.527808 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97494a7-db55-4ac7-a1b8-218b3cc0dd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:39.552473 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b97494a7-db55-4ac7-a1b8-218b3cc0dd7f None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:39.553835 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b97494a7-db55-4ac7-a1b8-218b3cc0dd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:39.561951 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1324/2622] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:39.566687 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97494a7-db55-4ac7-a1b8-218b3cc0dd7f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:39.575830 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97494a7-db55-4ac7-a1b8-218b3cc0dd7f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:39.580566 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1166c-c80d-4f64-a26c-81e4586e5974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:39.581150 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1166c-c80d-4f64-a26c-81e4586e5974 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:39.582033 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1166c-c80d-4f64-a26c-81e4586e5974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:39.582301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1166c-c80d-4f64-a26c-81e4586e5974 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:39.582972 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1166c-c80d-4f64-a26c-81e4586e5974 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:22:39.583269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1166c-c80d-4f64-a26c-81e4586e5974 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:39.584852 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1166c-c80d-4f64-a26c-81e4586e5974 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:39.596169 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1325/2623] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:22:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 17:22:39.601201 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1166c-c80d-4f64-a26c-81e4586e5974 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:39.611022 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1166c-c80d-4f64-a26c-81e4586e5974 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:39.616455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379baf4-16d3-4e14-9f45-2b175e846d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:39.617044 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379baf4-16d3-4e14-9f45-2b175e846d8f admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:39.617316 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379baf4-16d3-4e14-9f45-2b175e846d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:39.617584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379baf4-16d3-4e14-9f45-2b175e846d8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:39.618196 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2379baf4-16d3-4e14-9f45-2b175e846d8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:39.620158 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2379baf4-16d3-4e14-9f45-2b175e846d8f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:39.639462 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-28c78863-d135-4325-a1ac-3a5aaa8d25b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f201e9f902342279140cf9a6113845b', 'request_id': 'req-28c78863-d135-4325-a1ac-3a5aaa8d25b3'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:39.730803 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1326/2624] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:22:39 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:22:39.734969 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379baf4-16d3-4e14-9f45-2b175e846d8f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:39.735299 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-28c78863-d135-4325-a1ac-3a5aaa8d25b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f201e9f902342279140cf9a6113845b', 'request_id': 'req-28c78863-d135-4325-a1ac-3a5aaa8d25b3'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:39.742872 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-28c78863-d135-4325-a1ac-3a5aaa8d25b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f201e9f902342279140cf9a6113845b', 'request_id': 'req-28c78863-d135-4325-a1ac-3a5aaa8d25b3'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:39.753039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-28c78863-d135-4325-a1ac-3a5aaa8d25b3 admin admin] The token cache is being invalidate because project 4f201e9f902342279140cf9a6113845b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:22:39.754042 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-28c78863-d135-4325-a1ac-3a5aaa8d25b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:39.755045 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-28c78863-d135-4325-a1ac-3a5aaa8d25b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:39.756725 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1299/2625] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:22:39 2022] DELETE /identity/v3/projects/4f201e9f902342279140cf9a6113845b => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:22:39.760869 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c78863-d135-4325-a1ac-3a5aaa8d25b3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:39.762234 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-28c78863-d135-4325-a1ac-3a5aaa8d25b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:39.800225 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379baf4-16d3-4e14-9f45-2b175e846d8f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:39.806280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461df6fc-0381-410b-bb95-8e5046b18410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:39.807057 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461df6fc-0381-410b-bb95-8e5046b18410 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:39.807393 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461df6fc-0381-410b-bb95-8e5046b18410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:39.807904 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461df6fc-0381-410b-bb95-8e5046b18410 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:39.808603 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461df6fc-0381-410b-bb95-8e5046b18410 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:39.810741 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461df6fc-0381-410b-bb95-8e5046b18410 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:39.829906 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c78863-d135-4325-a1ac-3a5aaa8d25b3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:39.835214 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916868e7-3f6d-43f9-99c3-1ec25b11a7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:39.835835 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916868e7-3f6d-43f9-99c3-1ec25b11a7c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:39.836117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916868e7-3f6d-43f9-99c3-1ec25b11a7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:39.836386 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916868e7-3f6d-43f9-99c3-1ec25b11a7c2 admin admin] PATH_INFO: `/v3/users/4d2c3b26061347c8820cde695e1353af` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:39.837573 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916868e7-3f6d-43f9-99c3-1ec25b11a7c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d2c3b26061347c8820cde695e1353af)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:39.839532 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916868e7-3f6d-43f9-99c3-1ec25b11a7c2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:39.906556 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1327/2626] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:22:39 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:22:39.910699 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461df6fc-0381-410b-bb95-8e5046b18410 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:39.912319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-461df6fc-0381-410b-bb95-8e5046b18410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:39.924167 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461df6fc-0381-410b-bb95-8e5046b18410 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:39.930896 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a856a-1560-4743-82c6-bac1de0e7dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:39.931667 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a856a-1560-4743-82c6-bac1de0e7dee admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:39.931994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a856a-1560-4743-82c6-bac1de0e7dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:39.932378 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a856a-1560-4743-82c6-bac1de0e7dee admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:39.933208 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a856a-1560-4743-82c6-bac1de0e7dee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:39.935135 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a856a-1560-4743-82c6-bac1de0e7dee admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:39.946274 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1328/2627] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:22:39.950495 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a856a-1560-4743-82c6-bac1de0e7dee admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:39.959930 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a856a-1560-4743-82c6-bac1de0e7dee admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:39.964759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ad9b9-6108-4973-8793-97f071853de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:39.965326 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ad9b9-6108-4973-8793-97f071853de9 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:39.965608 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ad9b9-6108-4973-8793-97f071853de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:39.965877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ad9b9-6108-4973-8793-97f071853de9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:39.966561 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ad9b9-6108-4973-8793-97f071853de9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:39.968149 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ad9b9-6108-4973-8793-97f071853de9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:39.979160 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1329/2628] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:22:39.983183 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ad9b9-6108-4973-8793-97f071853de9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:39.993624 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ad9b9-6108-4973-8793-97f071853de9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:39.998324 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a050d-68ad-41d1-9afe-62d033adc8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:39.998974 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a050d-68ad-41d1-9afe-62d033adc8d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:39.999255 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a050d-68ad-41d1-9afe-62d033adc8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:39.999528 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a050d-68ad-41d1-9afe-62d033adc8d5 admin admin] PATH_INFO: `/v3/projects/61be6e0ac3b84dc1ab1143c68d078b50/users/3fea81b2cec04368922051382f33cc73/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:40.014122 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a050d-68ad-41d1-9afe-62d033adc8d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61be6e0ac3b84dc1ab1143c68d078b50, user_id=3fea81b2cec04368922051382f33cc73, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:40.016778 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a050d-68ad-41d1-9afe-62d033adc8d5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:40.085776 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-916868e7-3f6d-43f9-99c3-1ec25b11a7c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d2c3b26061347c8820cde695e1353af'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:40.101113 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1330/2629] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:39 2022] PUT /identity/v3/projects/61be6e0ac3b84dc1ab1143c68d078b50/users/3fea81b2cec04368922051382f33cc73/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:22:40.113596 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5e255-c518-470d-b6cd-89aaf3931524 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:40.113882 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5e255-c518-470d-b6cd-89aaf3931524 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:40.114155 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5e255-c518-470d-b6cd-89aaf3931524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:40.142876 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-98e5e255-c518-470d-b6cd-89aaf3931524 None None] MFA Rules not processed for user `3fea81b2cec04368922051382f33cc73`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:40.144219 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-98e5e255-c518-470d-b6cd-89aaf3931524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:40.151836 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-916868e7-3f6d-43f9-99c3-1ec25b11a7c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d2c3b26061347c8820cde695e1353af'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:40.166747 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-916868e7-3f6d-43f9-99c3-1ec25b11a7c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d2c3b26061347c8820cde695e1353af'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:40.167345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-916868e7-3f6d-43f9-99c3-1ec25b11a7c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d2c3b26061347c8820cde695e1353af'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:40.167958 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-916868e7-3f6d-43f9-99c3-1ec25b11a7c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d2c3b26061347c8820cde695e1353af'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:40.180935 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1300/2630] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:22:39 2022] DELETE /identity/v3/users/4d2c3b26061347c8820cde695e1353af => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:22:40.213025 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1331/2631] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:22:40.217606 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916868e7-3f6d-43f9-99c3-1ec25b11a7c2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:40.219157 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-916868e7-3f6d-43f9-99c3-1ec25b11a7c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:40.266469 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:40.290050 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916868e7-3f6d-43f9-99c3-1ec25b11a7c2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:40.294690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618fc5ff-4f5d-428a-989e-db99ac6128a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:40.295285 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618fc5ff-4f5d-428a-989e-db99ac6128a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:40.295558 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618fc5ff-4f5d-428a-989e-db99ac6128a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:40.295823 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618fc5ff-4f5d-428a-989e-db99ac6128a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:40.296437 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618fc5ff-4f5d-428a-989e-db99ac6128a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:40.298317 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618fc5ff-4f5d-428a-989e-db99ac6128a3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:40.324322 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:40.434381 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1301/2632] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:22:40 2022] POST /identity/v3/projects => generated 381 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:22:40.438434 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5e255-c518-470d-b6cd-89aaf3931524 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:40.480176 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618fc5ff-4f5d-428a-989e-db99ac6128a3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:40.482109 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-618fc5ff-4f5d-428a-989e-db99ac6128a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:40.506695 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5e255-c518-470d-b6cd-89aaf3931524 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:40.555975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618fc5ff-4f5d-428a-989e-db99ac6128a3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:40.560731 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678c40e-850c-4caa-bca2-e618b185dbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:40.561800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678c40e-850c-4caa-bca2-e618b185dbcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:40.562064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678c40e-850c-4caa-bca2-e618b185dbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:40.562334 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678c40e-850c-4caa-bca2-e618b185dbcf admin admin] PATH_INFO: `/v3/projects/51c96cb7c6c94223b1efbf2767117ca8` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:40.563530 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678c40e-850c-4caa-bca2-e618b185dbcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=51c96cb7c6c94223b1efbf2767117ca8)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:40.565464 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678c40e-850c-4caa-bca2-e618b185dbcf admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:40.567319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c38d6e-7b32-4de7-b661-f204426fd39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:40.567925 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c38d6e-7b32-4de7-b661-f204426fd39b admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:40.568202 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c38d6e-7b32-4de7-b661-f204426fd39b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:40.570888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c38d6e-7b32-4de7-b661-f204426fd39b admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:40.570888 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c38d6e-7b32-4de7-b661-f204426fd39b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:40.571098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c38d6e-7b32-4de7-b661-f204426fd39b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:40.632524 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1332/2633] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:22:40 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:22:40.642319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c38d6e-7b32-4de7-b661-f204426fd39b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:40.652766 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c38d6e-7b32-4de7-b661-f204426fd39b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:40.657282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305db45b-b62e-4db7-a7b5-aa4ccddd3a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:40.657865 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305db45b-b62e-4db7-a7b5-aa4ccddd3a7b admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:40.658138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305db45b-b62e-4db7-a7b5-aa4ccddd3a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:40.658435 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305db45b-b62e-4db7-a7b5-aa4ccddd3a7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:40.659124 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305db45b-b62e-4db7-a7b5-aa4ccddd3a7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:40.661361 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305db45b-b62e-4db7-a7b5-aa4ccddd3a7b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:40.671375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1333/2634] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:22:40.679774 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2678c40e-850c-4caa-bca2-e618b185dbcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51c96cb7c6c94223b1efbf2767117ca8', 'request_id': 'req-2678c40e-850c-4caa-bca2-e618b185dbcf'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:40.681652 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305db45b-b62e-4db7-a7b5-aa4ccddd3a7b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:40.683584 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:40.691565 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305db45b-b62e-4db7-a7b5-aa4ccddd3a7b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:40.748657 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85c0c8-e8be-4d75-a45c-ba4b6baac41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:40.749342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85c0c8-e8be-4d75-a45c-ba4b6baac41e admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:40.749670 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85c0c8-e8be-4d75-a45c-ba4b6baac41e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:40.749942 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85c0c8-e8be-4d75-a45c-ba4b6baac41e admin admin] PATH_INFO: `/v3/projects/983f707f54064ec0805602a6f0a7543c/users/5f568675ddf642f1b085e8616a925c76/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:40.763410 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2678c40e-850c-4caa-bca2-e618b185dbcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51c96cb7c6c94223b1efbf2767117ca8', 'request_id': 'req-2678c40e-850c-4caa-bca2-e618b185dbcf'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:40.765553 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85c0c8-e8be-4d75-a45c-ba4b6baac41e admin admin] RBAC: Authorizing `identity:create_grant(project_id=983f707f54064ec0805602a6f0a7543c, user_id=5f568675ddf642f1b085e8616a925c76, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:40.768256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85c0c8-e8be-4d75-a45c-ba4b6baac41e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:40.770862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2678c40e-850c-4caa-bca2-e618b185dbcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51c96cb7c6c94223b1efbf2767117ca8', 'request_id': 'req-2678c40e-850c-4caa-bca2-e618b185dbcf'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:40.782570 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2678c40e-850c-4caa-bca2-e618b185dbcf admin admin] The token cache is being invalidate because project 51c96cb7c6c94223b1efbf2767117ca8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:22:40.782860 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2678c40e-850c-4caa-bca2-e618b185dbcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:40.786192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2678c40e-850c-4caa-bca2-e618b185dbcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:40.786192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1302/2635] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:22:40 2022] DELETE /identity/v3/projects/51c96cb7c6c94223b1efbf2767117ca8 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:40.790542 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdabb0-c192-4c33-8494-cfb2c09d7183 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:40.790823 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdabb0-c192-4c33-8494-cfb2c09d7183 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:40.791093 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdabb0-c192-4c33-8494-cfb2c09d7183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:40.810316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-68cdabb0-c192-4c33-8494-cfb2c09d7183 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:40.811551 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-68cdabb0-c192-4c33-8494-cfb2c09d7183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:40.820048 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1303/2636] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:22:40.833071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517e462-8506-4841-bfd8-5983512d0054 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:40.833356 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517e462-8506-4841-bfd8-5983512d0054 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:40.833630 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517e462-8506-4841-bfd8-5983512d0054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:40.851067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:40.852959 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3517e462-8506-4841-bfd8-5983512d0054 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:40.854202 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3517e462-8506-4841-bfd8-5983512d0054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:40.857335 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1334/2637] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:40 2022] PUT /identity/v3/projects/983f707f54064ec0805602a6f0a7543c/users/5f568675ddf642f1b085e8616a925c76/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:22:40.863799 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85c0c8-e8be-4d75-a45c-ba4b6baac41e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:40.865766 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ca85c0c8-e8be-4d75-a45c-ba4b6baac41e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:40.923497 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1304/2638] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:22:40.932827 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3517e462-8506-4841-bfd8-5983512d0054 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:40.934071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85c0c8-e8be-4d75-a45c-ba4b6baac41e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:40.938791 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847aed5b-6b53-4f1e-b38d-b67fd19673fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:40.939473 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847aed5b-6b53-4f1e-b38d-b67fd19673fd admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:40.939747 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847aed5b-6b53-4f1e-b38d-b67fd19673fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:40.940015 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847aed5b-6b53-4f1e-b38d-b67fd19673fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:40.941215 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847aed5b-6b53-4f1e-b38d-b67fd19673fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:40.942724 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3517e462-8506-4841-bfd8-5983512d0054 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:40.942847 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847aed5b-6b53-4f1e-b38d-b67fd19673fd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:40.948050 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3967d-0d67-406d-9e08-b31482981392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:40.948648 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3967d-0d67-406d-9e08-b31482981392 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:40.948922 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3967d-0d67-406d-9e08-b31482981392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:40.949193 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3967d-0d67-406d-9e08-b31482981392 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:40.949899 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3967d-0d67-406d-9e08-b31482981392 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:22:40.950275 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3967d-0d67-406d-9e08-b31482981392 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:40.953425 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3967d-0d67-406d-9e08-b31482981392 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:40.953819 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1335/2639] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:40.958397 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847aed5b-6b53-4f1e-b38d-b67fd19673fd admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:40.964951 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1305/2640] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:22:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 17:22:40.968561 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847aed5b-6b53-4f1e-b38d-b67fd19673fd admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:40.969185 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3967d-0d67-406d-9e08-b31482981392 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:40.973617 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7818b3-f85a-43ed-9d75-bac64ef01913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:40.974272 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7818b3-f85a-43ed-9d75-bac64ef01913 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:40.974586 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7818b3-f85a-43ed-9d75-bac64ef01913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:40.974859 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7818b3-f85a-43ed-9d75-bac64ef01913 admin admin] PATH_INFO: `/v3/projects/983f707f54064ec0805602a6f0a7543c/users/5f568675ddf642f1b085e8616a925c76/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:40.976772 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7818b3-f85a-43ed-9d75-bac64ef01913 admin admin] RBAC: Authorizing `identity:create_grant(project_id=983f707f54064ec0805602a6f0a7543c, user_id=5f568675ddf642f1b085e8616a925c76, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:40.979630 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7818b3-f85a-43ed-9d75-bac64ef01913 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:40.981735 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3967d-0d67-406d-9e08-b31482981392 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:40.984498 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbaca05-3336-4e12-a977-98fe93820c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:40.985263 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbaca05-3336-4e12-a977-98fe93820c66 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:40.985536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbaca05-3336-4e12-a977-98fe93820c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:40.985840 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbaca05-3336-4e12-a977-98fe93820c66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:40.986490 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbaca05-3336-4e12-a977-98fe93820c66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:40.988508 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbaca05-3336-4e12-a977-98fe93820c66 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:40.991249 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1336/2641] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:40 2022] PUT /identity/v3/projects/983f707f54064ec0805602a6f0a7543c/users/5f568675ddf642f1b085e8616a925c76/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:22:41.001920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60014aa-677a-4c6b-8418-6c201d8f43ce None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:41.002226 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60014aa-677a-4c6b-8418-6c201d8f43ce None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:41.009606 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60014aa-677a-4c6b-8418-6c201d8f43ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:41.032614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e60014aa-677a-4c6b-8418-6c201d8f43ce None None] MFA Rules not processed for user `5f568675ddf642f1b085e8616a925c76`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:41.033817 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e60014aa-677a-4c6b-8418-6c201d8f43ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:41.102016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1337/2642] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:41 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:22:41.114282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60014aa-677a-4c6b-8418-6c201d8f43ce None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:41.115758 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e60014aa-677a-4c6b-8418-6c201d8f43ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:41.120595 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1306/2643] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:22:40 2022] POST /identity/v3/projects => generated 371 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:22:41.127417 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbaca05-3336-4e12-a977-98fe93820c66 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:41.189596 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbaca05-3336-4e12-a977-98fe93820c66 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:41.195123 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b7031-d5f5-4c1d-840b-0eae7fb703dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:41.195664 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b7031-d5f5-4c1d-840b-0eae7fb703dd admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:41.195939 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b7031-d5f5-4c1d-840b-0eae7fb703dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:41.196205 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b7031-d5f5-4c1d-840b-0eae7fb703dd admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:41.196695 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:41.196820 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b7031-d5f5-4c1d-840b-0eae7fb703dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:41.198773 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b7031-d5f5-4c1d-840b-0eae7fb703dd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:41.199253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60014aa-677a-4c6b-8418-6c201d8f43ce None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:41.248455 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94899558-7291-486f-a405-7dc153793e85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:41.249344 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94899558-7291-486f-a405-7dc153793e85 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:41.249947 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94899558-7291-486f-a405-7dc153793e85 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:41.250276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94899558-7291-486f-a405-7dc153793e85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:41.272481 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1307/2644] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:22:41 2022] POST /identity/v3/users => generated 286 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:22:41.276961 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b7031-d5f5-4c1d-840b-0eae7fb703dd admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:41.287231 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b7031-d5f5-4c1d-840b-0eae7fb703dd admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:41.291904 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfbdd26-db71-47a0-8632-7e41d420ee37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:41.292492 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfbdd26-db71-47a0-8632-7e41d420ee37 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:41.292763 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfbdd26-db71-47a0-8632-7e41d420ee37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:41.293030 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfbdd26-db71-47a0-8632-7e41d420ee37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:41.293701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfbdd26-db71-47a0-8632-7e41d420ee37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:41.295320 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfbdd26-db71-47a0-8632-7e41d420ee37 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:41.306055 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1308/2645] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:22:41.309934 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfbdd26-db71-47a0-8632-7e41d420ee37 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:41.316398 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94899558-7291-486f-a405-7dc153793e85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:41.317850 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94899558-7291-486f-a405-7dc153793e85 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:41.320298 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfbdd26-db71-47a0-8632-7e41d420ee37 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:41.324910 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652060a-6322-4554-95d8-6cb64f31d3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:41.325480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652060a-6322-4554-95d8-6cb64f31d3d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:41.325751 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652060a-6322-4554-95d8-6cb64f31d3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:41.326019 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652060a-6322-4554-95d8-6cb64f31d3d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:41.326870 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d652060a-6322-4554-95d8-6cb64f31d3d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:41.327633 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1338/2646] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:41 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:22:41.328692 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d652060a-6322-4554-95d8-6cb64f31d3d6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:41.339647 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1309/2647] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:22:41.343829 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94899558-7291-486f-a405-7dc153793e85 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:41.354441 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94899558-7291-486f-a405-7dc153793e85 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:41.361510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e40dd-382a-4f76-a0c8-1fc9c8eb8608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:41.362174 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e40dd-382a-4f76-a0c8-1fc9c8eb8608 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:41.363120 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e40dd-382a-4f76-a0c8-1fc9c8eb8608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:41.363405 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e40dd-382a-4f76-a0c8-1fc9c8eb8608 admin admin] PATH_INFO: `/v3/projects/23aa839ab67e40a8b174797f18688c83/users/b7b3e5b7b8a14d5c8f0c31504a900530/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:41.380770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e40dd-382a-4f76-a0c8-1fc9c8eb8608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23aa839ab67e40a8b174797f18688c83, user_id=b7b3e5b7b8a14d5c8f0c31504a900530, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:41.383904 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e40dd-382a-4f76-a0c8-1fc9c8eb8608 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:41.463086 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652060a-6322-4554-95d8-6cb64f31d3d6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:41.474453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652060a-6322-4554-95d8-6cb64f31d3d6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:41.478037 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92661ea7-d393-45d7-8680-e09382ae6776 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:41.478646 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92661ea7-d393-45d7-8680-e09382ae6776 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:41.478936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92661ea7-d393-45d7-8680-e09382ae6776 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:41.479183 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92661ea7-d393-45d7-8680-e09382ae6776 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:41.480870 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-92661ea7-d393-45d7-8680-e09382ae6776 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:41.483636 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1339/2648] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:41 2022] PUT /identity/v3/projects/23aa839ab67e40a8b174797f18688c83/users/b7b3e5b7b8a14d5c8f0c31504a900530/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:41.487624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:41.497629 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd955e3-7565-4ae2-bf93-12ce5a2bffae None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:41.497990 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd955e3-7565-4ae2-bf93-12ce5a2bffae None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:41.498331 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd955e3-7565-4ae2-bf93-12ce5a2bffae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:41.526830 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-edd955e3-7565-4ae2-bf93-12ce5a2bffae None None] MFA Rules not processed for user `b7b3e5b7b8a14d5c8f0c31504a900530`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:41.528180 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-edd955e3-7565-4ae2-bf93-12ce5a2bffae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:41.544049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92661ea7-d393-45d7-8680-e09382ae6776 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:41.545652 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92661ea7-d393-45d7-8680-e09382ae6776 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:41.555589 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1310/2649] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:22:41.599943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1340/2650] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:22:41.603854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92661ea7-d393-45d7-8680-e09382ae6776 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:41.616218 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92661ea7-d393-45d7-8680-e09382ae6776 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:41.673416 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a2498-fd90-4821-9c88-475799fac130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:41.674116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a2498-fd90-4821-9c88-475799fac130 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:41.674421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a2498-fd90-4821-9c88-475799fac130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:41.674689 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a2498-fd90-4821-9c88-475799fac130 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:41.676069 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a2498-fd90-4821-9c88-475799fac130 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:41.678058 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a2498-fd90-4821-9c88-475799fac130 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:41.771292 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1311/2651] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:22:41 2022] POST /identity/v3/projects => generated 371 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:22:41.775381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd955e3-7565-4ae2-bf93-12ce5a2bffae None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:41.786442 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd955e3-7565-4ae2-bf93-12ce5a2bffae None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:41.803376 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:41.843090 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82670d43-c91b-4766-bfba-63ebc3445409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:41.843692 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82670d43-c91b-4766-bfba-63ebc3445409 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:41.843973 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82670d43-c91b-4766-bfba-63ebc3445409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:41.844240 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82670d43-c91b-4766-bfba-63ebc3445409 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:41.844853 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82670d43-c91b-4766-bfba-63ebc3445409 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:41.846818 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82670d43-c91b-4766-bfba-63ebc3445409 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:41.998686 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1341/2652] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:22:41 2022] POST /identity/v3/users => generated 292 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:22:42.011260 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a2498-fd90-4821-9c88-475799fac130 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:42.022583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a2498-fd90-4821-9c88-475799fac130 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:42.027324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c8854-430d-4a2d-9dd2-c5d68822e130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:42.027908 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c8854-430d-4a2d-9dd2-c5d68822e130 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:42.028185 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c8854-430d-4a2d-9dd2-c5d68822e130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:42.028454 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c8854-430d-4a2d-9dd2-c5d68822e130 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:42.029089 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c8854-430d-4a2d-9dd2-c5d68822e130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:42.031147 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c8854-430d-4a2d-9dd2-c5d68822e130 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:42.043349 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1312/2653] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:42.047731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82670d43-c91b-4766-bfba-63ebc3445409 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:42.059981 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82670d43-c91b-4766-bfba-63ebc3445409 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:42.064977 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ca46f-75f0-4028-8569-f0b6c1fecc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:42.066646 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ca46f-75f0-4028-8569-f0b6c1fecc93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:42.066997 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ca46f-75f0-4028-8569-f0b6c1fecc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:42.067335 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ca46f-75f0-4028-8569-f0b6c1fecc93 admin admin] PATH_INFO: `/v3/projects/a2cd58944f4c420c84f33e424e6c4a15/users/f8f67b0ae477450d89988c97ec9416ec/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:42.084340 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8ca46f-75f0-4028-8569-f0b6c1fecc93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2cd58944f4c420c84f33e424e6c4a15, user_id=f8f67b0ae477450d89988c97ec9416ec, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:42.085677 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c8854-430d-4a2d-9dd2-c5d68822e130 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:42.087423 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8ca46f-75f0-4028-8569-f0b6c1fecc93 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:42.095761 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c8854-430d-4a2d-9dd2-c5d68822e130 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:42.100993 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3bed18-5cbf-4e02-a255-84337ee0559a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:42.101595 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3bed18-5cbf-4e02-a255-84337ee0559a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:42.101878 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3bed18-5cbf-4e02-a255-84337ee0559a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:42.102126 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3bed18-5cbf-4e02-a255-84337ee0559a admin admin] PATH_INFO: `/v3/users/3fea81b2cec04368922051382f33cc73` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:42.103284 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3bed18-5cbf-4e02-a255-84337ee0559a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fea81b2cec04368922051382f33cc73)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:42.105170 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3bed18-5cbf-4e02-a255-84337ee0559a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:42.202745 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1342/2654] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:42 2022] PUT /identity/v3/projects/a2cd58944f4c420c84f33e424e6c4a15/users/f8f67b0ae477450d89988c97ec9416ec/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:42.207235 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ca46f-75f0-4028-8569-f0b6c1fecc93 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:42.269582 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ca46f-75f0-4028-8569-f0b6c1fecc93 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:42.274199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2007f6-dc2f-4358-9104-8570cbb1a938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:42.274835 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2007f6-dc2f-4358-9104-8570cbb1a938 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:42.275122 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2007f6-dc2f-4358-9104-8570cbb1a938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:42.275395 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2007f6-dc2f-4358-9104-8570cbb1a938 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:42.276034 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2007f6-dc2f-4358-9104-8570cbb1a938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:42.277632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2007f6-dc2f-4358-9104-8570cbb1a938 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:42.287865 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1343/2655] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:22:42.292097 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2007f6-dc2f-4358-9104-8570cbb1a938 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:42.303052 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2007f6-dc2f-4358-9104-8570cbb1a938 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:42.308449 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64861a7-756f-4c94-b5b4-b60e457e20d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:42.309100 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64861a7-756f-4c94-b5b4-b60e457e20d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:42.309435 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64861a7-756f-4c94-b5b4-b60e457e20d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:42.309708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64861a7-756f-4c94-b5b4-b60e457e20d7 admin admin] PATH_INFO: `/v3/projects/a2cd58944f4c420c84f33e424e6c4a15/users/f8f67b0ae477450d89988c97ec9416ec/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:42.312171 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64861a7-756f-4c94-b5b4-b60e457e20d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2cd58944f4c420c84f33e424e6c4a15, user_id=f8f67b0ae477450d89988c97ec9416ec, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:42.315191 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64861a7-756f-4c94-b5b4-b60e457e20d7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:42.326732 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1344/2656] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:42 2022] PUT /identity/v3/projects/a2cd58944f4c420c84f33e424e6c4a15/users/f8f67b0ae477450d89988c97ec9416ec/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:22:42.343649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6677377-6807-41f9-a482-f6965a2814d9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:42.343940 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6677377-6807-41f9-a482-f6965a2814d9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:42.344212 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6677377-6807-41f9-a482-f6965a2814d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:42.371920 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7b3bed18-5cbf-4e02-a255-84337ee0559a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fea81b2cec04368922051382f33cc73'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:42.372464 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c6677377-6807-41f9-a482-f6965a2814d9 None None] MFA Rules not processed for user `f8f67b0ae477450d89988c97ec9416ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:42.373402 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c6677377-6807-41f9-a482-f6965a2814d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:42.436190 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:42.447258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1345/2657] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:42 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:22:42.448926 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7b3bed18-5cbf-4e02-a255-84337ee0559a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fea81b2cec04368922051382f33cc73'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:42.459197 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6677377-6807-41f9-a482-f6965a2814d9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:42.460730 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c6677377-6807-41f9-a482-f6965a2814d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:42.463536 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7b3bed18-5cbf-4e02-a255-84337ee0559a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fea81b2cec04368922051382f33cc73'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:42.464211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7b3bed18-5cbf-4e02-a255-84337ee0559a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fea81b2cec04368922051382f33cc73'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:42.465526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7b3bed18-5cbf-4e02-a255-84337ee0559a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fea81b2cec04368922051382f33cc73'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:42.470308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:42.483828 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1313/2658] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:22:42 2022] DELETE /identity/v3/users/3fea81b2cec04368922051382f33cc73 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:22:42.489347 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05525a5c-b22a-4055-b979-b769444d905f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:42.489625 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05525a5c-b22a-4055-b979-b769444d905f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:42.489897 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05525a5c-b22a-4055-b979-b769444d905f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:42.511642 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-05525a5c-b22a-4055-b979-b769444d905f None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:42.518231 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-05525a5c-b22a-4055-b979-b769444d905f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:42.522136 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6677377-6807-41f9-a482-f6965a2814d9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:42.573444 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebed567-b74f-411e-97dd-909251e98bf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:42.574206 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebed567-b74f-411e-97dd-909251e98bf8 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:42.574633 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebed567-b74f-411e-97dd-909251e98bf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:42.575095 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebed567-b74f-411e-97dd-909251e98bf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:42.577013 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0ebed567-b74f-411e-97dd-909251e98bf8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:42.577916 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1314/2659] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:22:42.599398 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05525a5c-b22a-4055-b979-b769444d905f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:42.601213 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-05525a5c-b22a-4055-b979-b769444d905f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:42.653985 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebed567-b74f-411e-97dd-909251e98bf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:42.655470 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebed567-b74f-411e-97dd-909251e98bf8 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:42.666111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05525a5c-b22a-4055-b979-b769444d905f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:42.666564 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1346/2660] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:42 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:22:42.669800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d5989-5390-4ef2-8ff4-aaa2c5ee7331 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:42.670433 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d5989-5390-4ef2-8ff4-aaa2c5ee7331 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:42.676006 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d5989-5390-4ef2-8ff4-aaa2c5ee7331 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:42.676006 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d5989-5390-4ef2-8ff4-aaa2c5ee7331 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:42.682310 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d5989-5390-4ef2-8ff4-aaa2c5ee7331 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:42.683775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d5989-5390-4ef2-8ff4-aaa2c5ee7331 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:42.694031 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1315/2661] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:22:43.268196 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebed567-b74f-411e-97dd-909251e98bf8 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:43.272708 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0ebed567-b74f-411e-97dd-909251e98bf8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:43.283616 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebed567-b74f-411e-97dd-909251e98bf8 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:43.288336 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a44d0-0433-438e-a563-f55dc25bec0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:43.288988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a44d0-0433-438e-a563-f55dc25bec0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:43.289268 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a44d0-0433-438e-a563-f55dc25bec0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:43.289541 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a44d0-0433-438e-a563-f55dc25bec0f admin admin] PATH_INFO: `/v3/projects/61be6e0ac3b84dc1ab1143c68d078b50` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:43.290690 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a44d0-0433-438e-a563-f55dc25bec0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=61be6e0ac3b84dc1ab1143c68d078b50)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:43.292601 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a44d0-0433-438e-a563-f55dc25bec0f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:43.445693 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ed6a44d0-0433-438e-a563-f55dc25bec0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61be6e0ac3b84dc1ab1143c68d078b50', 'request_id': 'req-ed6a44d0-0433-438e-a563-f55dc25bec0f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:43.452222 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ed6a44d0-0433-438e-a563-f55dc25bec0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61be6e0ac3b84dc1ab1143c68d078b50', 'request_id': 'req-ed6a44d0-0433-438e-a563-f55dc25bec0f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:43.453977 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:43.528862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ed6a44d0-0433-438e-a563-f55dc25bec0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61be6e0ac3b84dc1ab1143c68d078b50', 'request_id': 'req-ed6a44d0-0433-438e-a563-f55dc25bec0f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:43.542337 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ed6a44d0-0433-438e-a563-f55dc25bec0f admin admin] The token cache is being invalidate because project 61be6e0ac3b84dc1ab1143c68d078b50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:22:43.545610 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ed6a44d0-0433-438e-a563-f55dc25bec0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:43.545610 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ed6a44d0-0433-438e-a563-f55dc25bec0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:43.545610 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1347/2662] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:22:43 2022] DELETE /identity/v3/projects/61be6e0ac3b84dc1ab1143c68d078b50 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:43.549616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d5989-5390-4ef2-8ff4-aaa2c5ee7331 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:43.561155 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d5989-5390-4ef2-8ff4-aaa2c5ee7331 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:43.625924 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf39a6-6dff-4409-937a-e55cc06f0564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:43.631713 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf39a6-6dff-4409-937a-e55cc06f0564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:43.631713 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf39a6-6dff-4409-937a-e55cc06f0564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:43.631713 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf39a6-6dff-4409-937a-e55cc06f0564 admin admin] PATH_INFO: `/v3/users/5f568675ddf642f1b085e8616a925c76` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:43.631713 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf39a6-6dff-4409-937a-e55cc06f0564 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f568675ddf642f1b085e8616a925c76)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:43.631713 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf39a6-6dff-4409-937a-e55cc06f0564 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:43.811360 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:43.817295 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-07cf39a6-6dff-4409-937a-e55cc06f0564 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f568675ddf642f1b085e8616a925c76'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:43.861694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a44d0-0433-438e-a563-f55dc25bec0f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:43.863907 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ed6a44d0-0433-438e-a563-f55dc25bec0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:43.876992 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a44d0-0433-438e-a563-f55dc25bec0f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:43.882128 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53506ac-c954-44e7-a414-107d7ea64fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:43.882780 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53506ac-c954-44e7-a414-107d7ea64fc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:43.883059 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53506ac-c954-44e7-a414-107d7ea64fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:43.883330 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53506ac-c954-44e7-a414-107d7ea64fc8 admin admin] PATH_INFO: `/v3/users/b7b3e5b7b8a14d5c8f0c31504a900530` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:43.884459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53506ac-c954-44e7-a414-107d7ea64fc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7b3e5b7b8a14d5c8f0c31504a900530)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:43.886434 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53506ac-c954-44e7-a414-107d7ea64fc8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:43.912799 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-07cf39a6-6dff-4409-937a-e55cc06f0564 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f568675ddf642f1b085e8616a925c76'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:43.926200 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-07cf39a6-6dff-4409-937a-e55cc06f0564 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f568675ddf642f1b085e8616a925c76'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:43.926873 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-07cf39a6-6dff-4409-937a-e55cc06f0564 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f568675ddf642f1b085e8616a925c76'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:43.927552 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-07cf39a6-6dff-4409-937a-e55cc06f0564 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f568675ddf642f1b085e8616a925c76'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:43.939701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1316/2663] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:22:43 2022] DELETE /identity/v3/users/5f568675ddf642f1b085e8616a925c76 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:44.120916 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a53506ac-c954-44e7-a414-107d7ea64fc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7b3e5b7b8a14d5c8f0c31504a900530'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:44.209317 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a53506ac-c954-44e7-a414-107d7ea64fc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7b3e5b7b8a14d5c8f0c31504a900530'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:44.218153 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a53506ac-c954-44e7-a414-107d7ea64fc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7b3e5b7b8a14d5c8f0c31504a900530'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:44.234410 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a53506ac-c954-44e7-a414-107d7ea64fc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7b3e5b7b8a14d5c8f0c31504a900530'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:44.235180 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a53506ac-c954-44e7-a414-107d7ea64fc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7b3e5b7b8a14d5c8f0c31504a900530'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:44.241431 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1348/2664] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:22:43 2022] DELETE /identity/v3/users/b7b3e5b7b8a14d5c8f0c31504a900530 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:22:44.245882 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90024aaa-bad3-4bd3-98cc-20f9871b0825 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:44.246217 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90024aaa-bad3-4bd3-98cc-20f9871b0825 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:44.247427 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90024aaa-bad3-4bd3-98cc-20f9871b0825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:44.266858 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-90024aaa-bad3-4bd3-98cc-20f9871b0825 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:44.268298 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-90024aaa-bad3-4bd3-98cc-20f9871b0825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:44.332815 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1317/2665] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:44.342615 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53506ac-c954-44e7-a414-107d7ea64fc8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:44.344396 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a53506ac-c954-44e7-a414-107d7ea64fc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:44.405637 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53506ac-c954-44e7-a414-107d7ea64fc8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:44.409257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c9630-8b75-4491-8665-b8141d316458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:44.409934 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c9630-8b75-4491-8665-b8141d316458 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:44.410253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c9630-8b75-4491-8665-b8141d316458 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:44.410556 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c9630-8b75-4491-8665-b8141d316458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:44.424464 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672c9630-8b75-4491-8665-b8141d316458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:44.424644 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90024aaa-bad3-4bd3-98cc-20f9871b0825 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:44.426481 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672c9630-8b75-4491-8665-b8141d316458 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:44.427833 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-90024aaa-bad3-4bd3-98cc-20f9871b0825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:44.437757 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1349/2666] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:22:44.441862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90024aaa-bad3-4bd3-98cc-20f9871b0825 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:44.446855 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50efaa1-d910-4f74-a466-e3652b48b61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:44.447488 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50efaa1-d910-4f74-a466-e3652b48b61f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:44.447768 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50efaa1-d910-4f74-a466-e3652b48b61f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:44.448412 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50efaa1-d910-4f74-a466-e3652b48b61f admin admin] PATH_INFO: `/v3/projects/983f707f54064ec0805602a6f0a7543c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:44.456308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50efaa1-d910-4f74-a466-e3652b48b61f admin admin] RBAC: Authorizing `identity:delete_project(project_id=983f707f54064ec0805602a6f0a7543c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:44.458529 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50efaa1-d910-4f74-a466-e3652b48b61f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:44.593593 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d50efaa1-d910-4f74-a466-e3652b48b61f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '983f707f54064ec0805602a6f0a7543c', 'request_id': 'req-d50efaa1-d910-4f74-a466-e3652b48b61f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:44.666411 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d50efaa1-d910-4f74-a466-e3652b48b61f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '983f707f54064ec0805602a6f0a7543c', 'request_id': 'req-d50efaa1-d910-4f74-a466-e3652b48b61f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:44.673088 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d50efaa1-d910-4f74-a466-e3652b48b61f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '983f707f54064ec0805602a6f0a7543c', 'request_id': 'req-d50efaa1-d910-4f74-a466-e3652b48b61f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:44.685115 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d50efaa1-d910-4f74-a466-e3652b48b61f admin admin] The token cache is being invalidate because project 983f707f54064ec0805602a6f0a7543c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:22:44.685385 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d50efaa1-d910-4f74-a466-e3652b48b61f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:44.686148 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d50efaa1-d910-4f74-a466-e3652b48b61f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:44.688032 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1318/2667] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:22:44 2022] DELETE /identity/v3/projects/983f707f54064ec0805602a6f0a7543c => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:44.692549 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8296a40-33f6-40b3-aa5c-f5fb1a7199dc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:44.692847 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8296a40-33f6-40b3-aa5c-f5fb1a7199dc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:44.692996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8296a40-33f6-40b3-aa5c-f5fb1a7199dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:44.714867 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c8296a40-33f6-40b3-aa5c-f5fb1a7199dc None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:44.716083 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c8296a40-33f6-40b3-aa5c-f5fb1a7199dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:44.774529 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:44.782289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1350/2668] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:44.791336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35dbb9-c822-4705-8e38-3ffa515f54a8 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:44.791676 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35dbb9-c822-4705-8e38-3ffa515f54a8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:44.791871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35dbb9-c822-4705-8e38-3ffa515f54a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:44.813111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-9f35dbb9-c822-4705-8e38-3ffa515f54a8 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:44.814461 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9f35dbb9-c822-4705-8e38-3ffa515f54a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:44.822679 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1319/2669] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:44.827101 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8296a40-33f6-40b3-aa5c-f5fb1a7199dc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:44.837082 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8296a40-33f6-40b3-aa5c-f5fb1a7199dc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:44.841850 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d4e48-b948-4409-9ed1-5946a613f45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:44.842471 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d4e48-b948-4409-9ed1-5946a613f45e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:44.842748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d4e48-b948-4409-9ed1-5946a613f45e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:44.843013 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d4e48-b948-4409-9ed1-5946a613f45e admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:44.843649 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d4e48-b948-4409-9ed1-5946a613f45e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:22:44.843946 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d4e48-b948-4409-9ed1-5946a613f45e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:44.845558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d4e48-b948-4409-9ed1-5946a613f45e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:44.862284 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1351/2670] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:22:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:22:44.864167 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f35dbb9-c822-4705-8e38-3ffa515f54a8 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:44.873949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f35dbb9-c822-4705-8e38-3ffa515f54a8 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:44.878980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d9510-1d5c-4944-b4ad-9af724502efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:44.879766 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d9510-1d5c-4944-b4ad-9af724502efd admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:44.880223 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d9510-1d5c-4944-b4ad-9af724502efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:44.880496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d9510-1d5c-4944-b4ad-9af724502efd admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:44.881178 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d9510-1d5c-4944-b4ad-9af724502efd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:44.883686 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d9510-1d5c-4944-b4ad-9af724502efd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:44.897558 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d4e48-b948-4409-9ed1-5946a613f45e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:44.900466 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d50d4e48-b948-4409-9ed1-5946a613f45e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:44.917924 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d4e48-b948-4409-9ed1-5946a613f45e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:44.920766 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1b67d-9da5-4407-9594-57a1d0b69722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:44.921157 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1b67d-9da5-4407-9594-57a1d0b69722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:44.921445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1b67d-9da5-4407-9594-57a1d0b69722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:44.921718 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1b67d-9da5-4407-9594-57a1d0b69722 admin admin] PATH_INFO: `/v3/projects/23aa839ab67e40a8b174797f18688c83` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:44.922884 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c1b67d-9da5-4407-9594-57a1d0b69722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23aa839ab67e40a8b174797f18688c83)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:44.925978 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c1b67d-9da5-4407-9594-57a1d0b69722 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:45.034005 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1320/2671] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:22:44 2022] POST /identity/v3/projects => generated 373 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:22:45.038992 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d9510-1d5c-4944-b4ad-9af724502efd admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:45.044942 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d0c1b67d-9da5-4407-9594-57a1d0b69722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23aa839ab67e40a8b174797f18688c83', 'request_id': 'req-d0c1b67d-9da5-4407-9594-57a1d0b69722'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:45.052415 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d0c1b67d-9da5-4407-9594-57a1d0b69722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23aa839ab67e40a8b174797f18688c83', 'request_id': 'req-d0c1b67d-9da5-4407-9594-57a1d0b69722'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:45.052804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:45.100327 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d9510-1d5c-4944-b4ad-9af724502efd admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:45.106149 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1bcf42-bced-4dc2-8336-3b7a747a93cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:45.106783 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1bcf42-bced-4dc2-8336-3b7a747a93cf admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:45.107056 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1bcf42-bced-4dc2-8336-3b7a747a93cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:45.107319 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1bcf42-bced-4dc2-8336-3b7a747a93cf admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:45.107934 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1bcf42-bced-4dc2-8336-3b7a747a93cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:45.109829 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1bcf42-bced-4dc2-8336-3b7a747a93cf admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:45.157664 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d0c1b67d-9da5-4407-9594-57a1d0b69722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23aa839ab67e40a8b174797f18688c83', 'request_id': 'req-d0c1b67d-9da5-4407-9594-57a1d0b69722'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:45.168673 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d0c1b67d-9da5-4407-9594-57a1d0b69722 admin admin] The token cache is being invalidate because project 23aa839ab67e40a8b174797f18688c83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:22:45.168896 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d0c1b67d-9da5-4407-9594-57a1d0b69722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:45.169529 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d0c1b67d-9da5-4407-9594-57a1d0b69722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:45.171110 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1352/2672] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:22:44 2022] DELETE /identity/v3/projects/23aa839ab67e40a8b174797f18688c83 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:45.174862 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1b67d-9da5-4407-9594-57a1d0b69722 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:45.176727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d0c1b67d-9da5-4407-9594-57a1d0b69722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:45.243711 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1b67d-9da5-4407-9594-57a1d0b69722 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:45.248358 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401ffd4-db96-4e6c-9482-b2f205d2dafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:45.249006 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401ffd4-db96-4e6c-9482-b2f205d2dafa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:45.249386 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401ffd4-db96-4e6c-9482-b2f205d2dafa admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:45.249544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401ffd4-db96-4e6c-9482-b2f205d2dafa admin admin] PATH_INFO: `/v3/users/f8f67b0ae477450d89988c97ec9416ec` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:45.250705 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401ffd4-db96-4e6c-9482-b2f205d2dafa admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8f67b0ae477450d89988c97ec9416ec)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:45.252609 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401ffd4-db96-4e6c-9482-b2f205d2dafa admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:45.255926 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1321/2673] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:22:45 2022] POST /identity/v3/users => generated 287 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:22:45.260190 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1bcf42-bced-4dc2-8336-3b7a747a93cf admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:45.261932 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-eb1bcf42-bced-4dc2-8336-3b7a747a93cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:45.274907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1bcf42-bced-4dc2-8336-3b7a747a93cf admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:45.280815 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d01de-9831-4199-91f5-0e387903be4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:45.281873 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d01de-9831-4199-91f5-0e387903be4e admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:45.282117 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d01de-9831-4199-91f5-0e387903be4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:45.282786 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d01de-9831-4199-91f5-0e387903be4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:45.283071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d01de-9831-4199-91f5-0e387903be4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:45.284834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d01de-9831-4199-91f5-0e387903be4e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:45.294980 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1322/2674] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:45.298915 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d01de-9831-4199-91f5-0e387903be4e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:45.308481 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d01de-9831-4199-91f5-0e387903be4e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:45.313239 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23714055-ece2-4615-a991-e90b9362949f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:45.313811 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23714055-ece2-4615-a991-e90b9362949f admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:45.316326 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23714055-ece2-4615-a991-e90b9362949f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:45.316326 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23714055-ece2-4615-a991-e90b9362949f admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:45.316326 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23714055-ece2-4615-a991-e90b9362949f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:45.317082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23714055-ece2-4615-a991-e90b9362949f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:45.327446 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1323/2675] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:22:45.332060 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23714055-ece2-4615-a991-e90b9362949f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:45.341732 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23714055-ece2-4615-a991-e90b9362949f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:45.346402 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586f783-9f9d-4196-8b47-80b6c487c465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:45.350688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586f783-9f9d-4196-8b47-80b6c487c465 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:45.350688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586f783-9f9d-4196-8b47-80b6c487c465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:45.350688 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586f783-9f9d-4196-8b47-80b6c487c465 admin admin] PATH_INFO: `/v3/projects/7feac9a147e3467a87882322f37821f4/users/cd18b8117c724c72a36348bcc5aaae7f/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:45.362048 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3586f783-9f9d-4196-8b47-80b6c487c465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7feac9a147e3467a87882322f37821f4, user_id=cd18b8117c724c72a36348bcc5aaae7f, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:45.364873 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3586f783-9f9d-4196-8b47-80b6c487c465 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:45.438255 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1324/2676] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:45 2022] PUT /identity/v3/projects/7feac9a147e3467a87882322f37821f4/users/cd18b8117c724c72a36348bcc5aaae7f/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:22:45.443994 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19caa82b-d17b-4b69-bce4-96b39a3df062 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:45.444261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19caa82b-d17b-4b69-bce4-96b39a3df062 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:45.444526 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19caa82b-d17b-4b69-bce4-96b39a3df062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:45.464178 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-19caa82b-d17b-4b69-bce4-96b39a3df062 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:45.465377 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-19caa82b-d17b-4b69-bce4-96b39a3df062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:45.523452 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1325/2677] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:22:45.536690 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19caa82b-d17b-4b69-bce4-96b39a3df062 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:45.538096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-19caa82b-d17b-4b69-bce4-96b39a3df062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:45.560901 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c401ffd4-db96-4e6c-9482-b2f205d2dafa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8f67b0ae477450d89988c97ec9416ec'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:45.599742 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19caa82b-d17b-4b69-bce4-96b39a3df062 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:45.603330 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e04939-1549-4c73-b618-ac85efaa79a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:45.603932 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e04939-1549-4c73-b618-ac85efaa79a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:45.604198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e04939-1549-4c73-b618-ac85efaa79a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:45.604464 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e04939-1549-4c73-b618-ac85efaa79a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:45.615165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e04939-1549-4c73-b618-ac85efaa79a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:45.616604 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e04939-1549-4c73-b618-ac85efaa79a2 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:45.620705 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:45.626986 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1326/2678] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:22:45.648711 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c401ffd4-db96-4e6c-9482-b2f205d2dafa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8f67b0ae477450d89988c97ec9416ec'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:45.656413 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c401ffd4-db96-4e6c-9482-b2f205d2dafa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8f67b0ae477450d89988c97ec9416ec'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:45.661809 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:45.668738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c401ffd4-db96-4e6c-9482-b2f205d2dafa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8f67b0ae477450d89988c97ec9416ec'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:45.669473 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c401ffd4-db96-4e6c-9482-b2f205d2dafa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8f67b0ae477450d89988c97ec9416ec'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:45.676115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1353/2679] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:22:45 2022] DELETE /identity/v3/users/f8f67b0ae477450d89988c97ec9416ec => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:22:46.125520 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e04939-1549-4c73-b618-ac85efaa79a2 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:46.144776 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e04939-1549-4c73-b618-ac85efaa79a2 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:46.205711 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3c2e7-2133-41b7-831f-72b6b33140e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:46.207456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3c2e7-2133-41b7-831f-72b6b33140e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:46.207737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3c2e7-2133-41b7-831f-72b6b33140e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:46.208004 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3c2e7-2133-41b7-831f-72b6b33140e1 admin admin] PATH_INFO: `/v3/projects/a2cd58944f4c420c84f33e424e6c4a15` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:46.209158 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d3c2e7-2133-41b7-831f-72b6b33140e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2cd58944f4c420c84f33e424e6c4a15)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:46.211130 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d3c2e7-2133-41b7-831f-72b6b33140e1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:46.372132 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b2d3c2e7-2133-41b7-831f-72b6b33140e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2cd58944f4c420c84f33e424e6c4a15', 'request_id': 'req-b2d3c2e7-2133-41b7-831f-72b6b33140e1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:46.477633 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b2d3c2e7-2133-41b7-831f-72b6b33140e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2cd58944f4c420c84f33e424e6c4a15', 'request_id': 'req-b2d3c2e7-2133-41b7-831f-72b6b33140e1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:46.484864 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b2d3c2e7-2133-41b7-831f-72b6b33140e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2cd58944f4c420c84f33e424e6c4a15', 'request_id': 'req-b2d3c2e7-2133-41b7-831f-72b6b33140e1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:46.499705 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b2d3c2e7-2133-41b7-831f-72b6b33140e1 admin admin] The token cache is being invalidate because project a2cd58944f4c420c84f33e424e6c4a15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:22:46.499974 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b2d3c2e7-2133-41b7-831f-72b6b33140e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:46.500629 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b2d3c2e7-2133-41b7-831f-72b6b33140e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:46.502410 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1327/2680] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:22:46 2022] DELETE /identity/v3/projects/a2cd58944f4c420c84f33e424e6c4a15 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:46.508313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a6947-00bb-4993-80f6-b1850276b0c5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:46.508699 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a6947-00bb-4993-80f6-b1850276b0c5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:46.509049 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a6947-00bb-4993-80f6-b1850276b0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:46.528812 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-714a6947-00bb-4993-80f6-b1850276b0c5 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:46.530019 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-714a6947-00bb-4993-80f6-b1850276b0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:46.593938 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1354/2681] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:46.602610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00aeec-37cd-4524-8e59-649a17db54d2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:46.603079 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00aeec-37cd-4524-8e59-649a17db54d2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:46.603202 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00aeec-37cd-4524-8e59-649a17db54d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:46.625501 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2c00aeec-37cd-4524-8e59-649a17db54d2 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:46.626790 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2c00aeec-37cd-4524-8e59-649a17db54d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:46.636131 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1328/2682] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:22:46.640600 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a6947-00bb-4993-80f6-b1850276b0c5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:46.650861 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a6947-00bb-4993-80f6-b1850276b0c5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:46.656096 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83dba98-8a52-4827-bb1a-e97c51d1ef14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:46.656713 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83dba98-8a52-4827-bb1a-e97c51d1ef14 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:46.656995 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83dba98-8a52-4827-bb1a-e97c51d1ef14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:46.657269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83dba98-8a52-4827-bb1a-e97c51d1ef14 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:46.657979 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83dba98-8a52-4827-bb1a-e97c51d1ef14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:22:46.658291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83dba98-8a52-4827-bb1a-e97c51d1ef14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:46.660140 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83dba98-8a52-4827-bb1a-e97c51d1ef14 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:46.672234 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1355/2683] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:22:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 17:22:46.677346 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00aeec-37cd-4524-8e59-649a17db54d2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:46.686871 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00aeec-37cd-4524-8e59-649a17db54d2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:46.692258 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19f50d-2e83-4576-a1b7-94b1368a3224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:46.692839 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19f50d-2e83-4576-a1b7-94b1368a3224 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:46.693180 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19f50d-2e83-4576-a1b7-94b1368a3224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:46.693448 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19f50d-2e83-4576-a1b7-94b1368a3224 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:46.694077 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19f50d-2e83-4576-a1b7-94b1368a3224 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:46.696001 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19f50d-2e83-4576-a1b7-94b1368a3224 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:46.869291 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1329/2684] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:22:46 2022] POST /identity/v3/projects => generated 387 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:22:46.873795 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83dba98-8a52-4827-bb1a-e97c51d1ef14 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:46.884441 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83dba98-8a52-4827-bb1a-e97c51d1ef14 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:46.946837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26295752-06f2-47d6-9de5-3ce4b3f4f5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:46.951145 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26295752-06f2-47d6-9de5-3ce4b3f4f5de admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:46.951145 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26295752-06f2-47d6-9de5-3ce4b3f4f5de admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:46.951145 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26295752-06f2-47d6-9de5-3ce4b3f4f5de admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:46.951145 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26295752-06f2-47d6-9de5-3ce4b3f4f5de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:46.951145 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26295752-06f2-47d6-9de5-3ce4b3f4f5de admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:47.052522 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1356/2685] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:22:46 2022] POST /identity/v3/users => generated 294 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:22:47.056698 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19f50d-2e83-4576-a1b7-94b1368a3224 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:47.068125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19f50d-2e83-4576-a1b7-94b1368a3224 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:47.073945 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7415932-22c1-4133-b04d-53d334092e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:47.074598 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7415932-22c1-4133-b04d-53d334092e03 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:47.074874 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7415932-22c1-4133-b04d-53d334092e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:47.075157 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7415932-22c1-4133-b04d-53d334092e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:47.075789 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7415932-22c1-4133-b04d-53d334092e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:47.077444 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7415932-22c1-4133-b04d-53d334092e03 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:47.088565 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1330/2686] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:47.092737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26295752-06f2-47d6-9de5-3ce4b3f4f5de admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:47.104692 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26295752-06f2-47d6-9de5-3ce4b3f4f5de admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:47.109599 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df86dd-8e40-4600-9e26-ea3123d1f686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:47.110421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df86dd-8e40-4600-9e26-ea3123d1f686 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:47.115071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df86dd-8e40-4600-9e26-ea3123d1f686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:47.115071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df86dd-8e40-4600-9e26-ea3123d1f686 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:47.115071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df86dd-8e40-4600-9e26-ea3123d1f686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:47.115071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df86dd-8e40-4600-9e26-ea3123d1f686 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:47.129320 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1357/2687] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:22:47.130659 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7415932-22c1-4133-b04d-53d334092e03 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:47.141702 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7415932-22c1-4133-b04d-53d334092e03 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:47.147477 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665550c-f84d-41dc-850e-b37116ca8394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:47.152232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665550c-f84d-41dc-850e-b37116ca8394 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:47.152232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665550c-f84d-41dc-850e-b37116ca8394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:47.152232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665550c-f84d-41dc-850e-b37116ca8394 admin admin] PATH_INFO: `/v3/projects/41b17d07b4e044faa078e6001fd9b49c/users/9c945ff72665481b9c9506a3cca6cfde/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:47.165624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4665550c-f84d-41dc-850e-b37116ca8394 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41b17d07b4e044faa078e6001fd9b49c, user_id=9c945ff72665481b9c9506a3cca6cfde, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:47.168336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4665550c-f84d-41dc-850e-b37116ca8394 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:47.246410 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1331/2688] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:47 2022] PUT /identity/v3/projects/41b17d07b4e044faa078e6001fd9b49c/users/9c945ff72665481b9c9506a3cca6cfde/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:47.246656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70eedaf-464e-417d-8fb6-d916238fb0ff None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:47.246656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70eedaf-464e-417d-8fb6-d916238fb0ff None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:47.247070 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70eedaf-464e-417d-8fb6-d916238fb0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:47.258045 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0837ee-c945-46ac-a195-efa6f79d928e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:47.258376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0837ee-c945-46ac-a195-efa6f79d928e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:47.258621 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0837ee-c945-46ac-a195-efa6f79d928e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:47.283181 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f70eedaf-464e-417d-8fb6-d916238fb0ff None None] MFA Rules not processed for user `cd18b8117c724c72a36348bcc5aaae7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:47.285293 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f70eedaf-464e-417d-8fb6-d916238fb0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:47.291274 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ea0837ee-c945-46ac-a195-efa6f79d928e None None] MFA Rules not processed for user `9c945ff72665481b9c9506a3cca6cfde`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:47.292571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ea0837ee-c945-46ac-a195-efa6f79d928e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:47.307375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:47.340002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:47.360030 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1358/2689] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:22:47.366342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70eedaf-464e-417d-8fb6-d916238fb0ff None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:47.367340 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f70eedaf-464e-417d-8fb6-d916238fb0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:47.381659 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1332/2690] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:22:47.386469 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0837ee-c945-46ac-a195-efa6f79d928e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:47.448447 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70eedaf-464e-417d-8fb6-d916238fb0ff None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:47.453267 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab756a-ee7b-44ed-ad70-51a982292212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:47.453941 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab756a-ee7b-44ed-ad70-51a982292212 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:47.454291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab756a-ee7b-44ed-ad70-51a982292212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:47.454680 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab756a-ee7b-44ed-ad70-51a982292212 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:47.455170 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0837ee-c945-46ac-a195-efa6f79d928e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:47.455437 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ab756a-ee7b-44ed-ad70-51a982292212 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:47.458301 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ab756a-ee7b-44ed-ad70-51a982292212 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:47.461324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd96a83-c067-41f4-9955-b401e7389e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:47.461959 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd96a83-c067-41f4-9955-b401e7389e8d admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:47.462707 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd96a83-c067-41f4-9955-b401e7389e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:47.462992 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd96a83-c067-41f4-9955-b401e7389e8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:47.463728 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd96a83-c067-41f4-9955-b401e7389e8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:47.465610 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd96a83-c067-41f4-9955-b401e7389e8d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:47.592512 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1359/2691] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:22:47 2022] POST /identity/v3/projects => generated 373 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:22:47.593353 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab756a-ee7b-44ed-ad70-51a982292212 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:47.595658 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c5ab756a-ee7b-44ed-ad70-51a982292212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:47.616868 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1333/2692] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:22:47 2022] POST /identity/v3/projects => generated 387 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:22:47.620820 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd96a83-c067-41f4-9955-b401e7389e8d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:47.663352 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab756a-ee7b-44ed-ad70-51a982292212 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:47.684457 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd96a83-c067-41f4-9955-b401e7389e8d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:47.689637 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda20a5b-ab49-4594-894e-f06851dd196a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:47.690244 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda20a5b-ab49-4594-894e-f06851dd196a admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:47.691061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda20a5b-ab49-4594-894e-f06851dd196a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:47.691334 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda20a5b-ab49-4594-894e-f06851dd196a admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:47.691957 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda20a5b-ab49-4594-894e-f06851dd196a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:47.693932 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda20a5b-ab49-4594-894e-f06851dd196a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:47.722423 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc1b8b-fe7f-4620-9f4e-df9f2ecfa316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:47.723091 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc1b8b-fe7f-4620-9f4e-df9f2ecfa316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:47.723544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc1b8b-fe7f-4620-9f4e-df9f2ecfa316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:47.723827 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc1b8b-fe7f-4620-9f4e-df9f2ecfa316 admin admin] PATH_INFO: `/v3/users/0897fe87870e464cab17d9592786c6d8` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:47.724962 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc1b8b-fe7f-4620-9f4e-df9f2ecfa316 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0897fe87870e464cab17d9592786c6d8)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:47.726928 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc1b8b-fe7f-4620-9f4e-df9f2ecfa316 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:47.743299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:47.785722 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1334/2693] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:22:47 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:22:47.786617 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda20a5b-ab49-4594-894e-f06851dd196a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:47.796050 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda20a5b-ab49-4594-894e-f06851dd196a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:47.800926 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10661a-632a-49ac-8ad5-e623c486df9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:47.807934 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10661a-632a-49ac-8ad5-e623c486df9a admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:47.807934 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10661a-632a-49ac-8ad5-e623c486df9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:47.807934 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10661a-632a-49ac-8ad5-e623c486df9a admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:47.807934 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10661a-632a-49ac-8ad5-e623c486df9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:47.807934 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10661a-632a-49ac-8ad5-e623c486df9a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:47.958278 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1335/2694] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:22:47 2022] POST /identity/v3/users => generated 300 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:22:47.959223 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10661a-632a-49ac-8ad5-e623c486df9a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:47.969105 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10661a-632a-49ac-8ad5-e623c486df9a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:47.974044 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a333d-2bd3-4df4-9356-056b1a532a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:47.974657 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a333d-2bd3-4df4-9356-056b1a532a7a admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:47.974939 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a333d-2bd3-4df4-9356-056b1a532a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:47.975214 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a333d-2bd3-4df4-9356-056b1a532a7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:47.975842 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a333d-2bd3-4df4-9356-056b1a532a7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:47.977421 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a333d-2bd3-4df4-9356-056b1a532a7a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:47.982608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:47.988721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1336/2695] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:47.989673 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a333d-2bd3-4df4-9356-056b1a532a7a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:47.999776 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a333d-2bd3-4df4-9356-056b1a532a7a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:48.005096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82604659-6df1-47f4-b006-81cada0d8dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:48.005669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82604659-6df1-47f4-b006-81cada0d8dba admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:48.005943 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82604659-6df1-47f4-b006-81cada0d8dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:48.006210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82604659-6df1-47f4-b006-81cada0d8dba admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:48.007131 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82604659-6df1-47f4-b006-81cada0d8dba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:48.008721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82604659-6df1-47f4-b006-81cada0d8dba admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:48.018407 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1337/2696] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:48.020142 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82604659-6df1-47f4-b006-81cada0d8dba admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:48.029701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82604659-6df1-47f4-b006-81cada0d8dba admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:48.035699 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564057b-2ece-472f-9bce-309f4fee3dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:48.036236 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9bbc1b8b-fe7f-4620-9f4e-df9f2ecfa316 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0897fe87870e464cab17d9592786c6d8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:48.036379 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564057b-2ece-472f-9bce-309f4fee3dde admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:48.036656 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564057b-2ece-472f-9bce-309f4fee3dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:48.037051 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564057b-2ece-472f-9bce-309f4fee3dde admin admin] PATH_INFO: `/v3/projects/0e92e33a696c4473b35334d82030528f/users/59800960b5a44ca78d5cb67fee10d366/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:48.052231 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564057b-2ece-472f-9bce-309f4fee3dde admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e92e33a696c4473b35334d82030528f, user_id=59800960b5a44ca78d5cb67fee10d366, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:48.054985 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564057b-2ece-472f-9bce-309f4fee3dde admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:48.140335 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9bbc1b8b-fe7f-4620-9f4e-df9f2ecfa316 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0897fe87870e464cab17d9592786c6d8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:48.148100 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9bbc1b8b-fe7f-4620-9f4e-df9f2ecfa316 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0897fe87870e464cab17d9592786c6d8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:48.159994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9bbc1b8b-fe7f-4620-9f4e-df9f2ecfa316 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0897fe87870e464cab17d9592786c6d8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:48.160678 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9bbc1b8b-fe7f-4620-9f4e-df9f2ecfa316 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0897fe87870e464cab17d9592786c6d8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:48.166684 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1360/2697] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:22:47 2022] DELETE /identity/v3/users/0897fe87870e464cab17d9592786c6d8 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:48.167485 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc1b8b-fe7f-4620-9f4e-df9f2ecfa316 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:48.168799 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9bbc1b8b-fe7f-4620-9f4e-df9f2ecfa316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:48.181587 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1338/2698] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:48 2022] PUT /identity/v3/projects/0e92e33a696c4473b35334d82030528f/users/59800960b5a44ca78d5cb67fee10d366/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:48.184625 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6b030-9a6a-4a7c-b4ff-ffb5258fa1fc None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:48.184625 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6b030-9a6a-4a7c-b4ff-ffb5258fa1fc None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:48.184625 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6b030-9a6a-4a7c-b4ff-ffb5258fa1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:48.203665 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4ca6b030-9a6a-4a7c-b4ff-ffb5258fa1fc None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:48.204967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4ca6b030-9a6a-4a7c-b4ff-ffb5258fa1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:48.236454 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc1b8b-fe7f-4620-9f4e-df9f2ecfa316 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:48.266547 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1339/2699] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:48.267473 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6b030-9a6a-4a7c-b4ff-ffb5258fa1fc None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:48.269988 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4ca6b030-9a6a-4a7c-b4ff-ffb5258fa1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:48.281811 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6b030-9a6a-4a7c-b4ff-ffb5258fa1fc None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:48.287271 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aaa108-2fb4-4ae9-bd24-dc6d7b4a841d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:48.287921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aaa108-2fb4-4ae9-bd24-dc6d7b4a841d admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:48.288317 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aaa108-2fb4-4ae9-bd24-dc6d7b4a841d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:48.288729 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aaa108-2fb4-4ae9-bd24-dc6d7b4a841d admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:48.289237 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aaa108-2fb4-4ae9-bd24-dc6d7b4a841d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:48.290872 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aaa108-2fb4-4ae9-bd24-dc6d7b4a841d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:48.298105 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc516b11-509b-4313-ac63-8b85a001b88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:48.298784 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc516b11-509b-4313-ac63-8b85a001b88b admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:48.299058 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc516b11-509b-4313-ac63-8b85a001b88b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:48.299323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc516b11-509b-4313-ac63-8b85a001b88b admin admin] PATH_INFO: `/v3/projects/0e610a95c6044556b7a7aa71f567b1f2/users/4662b07b5bf04268828f84412ac6cbc7/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:48.300695 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1340/2700] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:48.305078 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aaa108-2fb4-4ae9-bd24-dc6d7b4a841d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:48.305078 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-22aaa108-2fb4-4ae9-bd24-dc6d7b4a841d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:48.314720 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc516b11-509b-4313-ac63-8b85a001b88b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e610a95c6044556b7a7aa71f567b1f2, user_id=4662b07b5bf04268828f84412ac6cbc7, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:48.317681 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc516b11-509b-4313-ac63-8b85a001b88b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:48.362028 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aaa108-2fb4-4ae9-bd24-dc6d7b4a841d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:48.366324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a7085-4244-479c-a561-5a58f8af0fc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:48.366987 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a7085-4244-479c-a561-5a58f8af0fc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:48.367265 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a7085-4244-479c-a561-5a58f8af0fc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:48.367540 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a7085-4244-479c-a561-5a58f8af0fc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:48.377019 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a7085-4244-479c-a561-5a58f8af0fc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:48.378537 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a7085-4244-479c-a561-5a58f8af0fc1 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:48.388435 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1341/2701] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:22:48.391116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a7085-4244-479c-a561-5a58f8af0fc1 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:48.398286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a7085-4244-479c-a561-5a58f8af0fc1 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:48.403192 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb599cd8-f6dd-4de1-8fbd-1e9aee2f98f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:48.403835 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb599cd8-f6dd-4de1-8fbd-1e9aee2f98f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:48.404106 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb599cd8-f6dd-4de1-8fbd-1e9aee2f98f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:48.404375 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb599cd8-f6dd-4de1-8fbd-1e9aee2f98f7 admin admin] PATH_INFO: `/v3/projects/0e92e33a696c4473b35334d82030528f/users/59800960b5a44ca78d5cb67fee10d366/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:48.406316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb599cd8-f6dd-4de1-8fbd-1e9aee2f98f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e92e33a696c4473b35334d82030528f, user_id=59800960b5a44ca78d5cb67fee10d366, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:48.409165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb599cd8-f6dd-4de1-8fbd-1e9aee2f98f7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:48.416024 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1361/2702] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:48 2022] PUT /identity/v3/projects/0e610a95c6044556b7a7aa71f567b1f2/users/4662b07b5bf04268828f84412ac6cbc7/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:48.420274 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc516b11-509b-4313-ac63-8b85a001b88b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:48.423539 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1342/2703] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:48 2022] PUT /identity/v3/projects/0e92e33a696c4473b35334d82030528f/users/59800960b5a44ca78d5cb67fee10d366/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:48.459274 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:48.497834 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc516b11-509b-4313-ac63-8b85a001b88b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:48.554238 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005fa72-ddbb-49e5-b744-a80d2c8c593b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:48.554878 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005fa72-ddbb-49e5-b744-a80d2c8c593b admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:48.555155 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005fa72-ddbb-49e5-b744-a80d2c8c593b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:48.555427 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005fa72-ddbb-49e5-b744-a80d2c8c593b admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:48.556423 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4005fa72-ddbb-49e5-b744-a80d2c8c593b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:48.557875 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4005fa72-ddbb-49e5-b744-a80d2c8c593b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:48.568937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1362/2704] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:48.573307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb599cd8-f6dd-4de1-8fbd-1e9aee2f98f7 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:48.584508 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb599cd8-f6dd-4de1-8fbd-1e9aee2f98f7 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:48.590516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec3ac2-230c-495c-9119-66d9d114f160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:48.590820 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec3ac2-230c-495c-9119-66d9d114f160 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:48.591126 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec3ac2-230c-495c-9119-66d9d114f160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:48.591353 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec3ac2-230c-495c-9119-66d9d114f160 admin admin] PATH_INFO: `/v3/projects/0e610a95c6044556b7a7aa71f567b1f2/users/4662b07b5bf04268828f84412ac6cbc7/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:48.593473 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec3ac2-230c-495c-9119-66d9d114f160 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e610a95c6044556b7a7aa71f567b1f2, user_id=4662b07b5bf04268828f84412ac6cbc7, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:48.596263 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec3ac2-230c-495c-9119-66d9d114f160 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:48.611339 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1343/2705] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:48 2022] PUT /identity/v3/projects/0e610a95c6044556b7a7aa71f567b1f2/users/4662b07b5bf04268828f84412ac6cbc7/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:48.627823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e6458-d038-47e1-a65f-9eac97bc3bae None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:48.628294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e6458-d038-47e1-a65f-9eac97bc3bae None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:48.628396 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e6458-d038-47e1-a65f-9eac97bc3bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:48.659117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-dc0e6458-d038-47e1-a65f-9eac97bc3bae None None] MFA Rules not processed for user `4662b07b5bf04268828f84412ac6cbc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:48.661011 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-dc0e6458-d038-47e1-a65f-9eac97bc3bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:48.733785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1363/2706] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:48.745430 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec3ac2-230c-495c-9119-66d9d114f160 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:48.747240 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-84ec3ac2-230c-495c-9119-66d9d114f160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:48.784905 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:48.819239 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec3ac2-230c-495c-9119-66d9d114f160 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:48.822867 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534075e0-4eca-48cf-9be9-55b722726d5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:48.823462 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534075e0-4eca-48cf-9be9-55b722726d5c service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:48.823732 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534075e0-4eca-48cf-9be9-55b722726d5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:48.824000 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534075e0-4eca-48cf-9be9-55b722726d5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:48.833967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534075e0-4eca-48cf-9be9-55b722726d5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:48.835459 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534075e0-4eca-48cf-9be9-55b722726d5c service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:48.845465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1344/2707] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:22:48.897887 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e6458-d038-47e1-a65f-9eac97bc3bae None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:48.899426 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-dc0e6458-d038-47e1-a65f-9eac97bc3bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:48.916455 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534075e0-4eca-48cf-9be9-55b722726d5c service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:48.969749 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:48.984438 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e6458-d038-47e1-a65f-9eac97bc3bae None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:48.989313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de44d7b7-d1ba-45a0-9469-bacc464837a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:48.990123 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44d7b7-d1ba-45a0-9469-bacc464837a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:48.990435 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44d7b7-d1ba-45a0-9469-bacc464837a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:48.990709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44d7b7-d1ba-45a0-9469-bacc464837a3 admin admin] PATH_INFO: `/v3/projects/2c9ea9542e1b4d95b9af64192e8544f1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:48.991825 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de44d7b7-d1ba-45a0-9469-bacc464837a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c9ea9542e1b4d95b9af64192e8544f1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:48.993731 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de44d7b7-d1ba-45a0-9469-bacc464837a3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:48.996721 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534075e0-4eca-48cf-9be9-55b722726d5c service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:49.003227 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128129e2-12e0-4bea-8f84-84098720361a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:49.003227 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128129e2-12e0-4bea-8f84-84098720361a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:49.003479 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128129e2-12e0-4bea-8f84-84098720361a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:49.003705 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128129e2-12e0-4bea-8f84-84098720361a admin admin] PATH_INFO: `/v3/users/9c945ff72665481b9c9506a3cca6cfde` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:49.005522 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128129e2-12e0-4bea-8f84-84098720361a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c945ff72665481b9c9506a3cca6cfde)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:49.007807 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128129e2-12e0-4bea-8f84-84098720361a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:49.243256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-de44d7b7-d1ba-45a0-9469-bacc464837a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c9ea9542e1b4d95b9af64192e8544f1', 'request_id': 'req-de44d7b7-d1ba-45a0-9469-bacc464837a3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:49.245052 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-128129e2-12e0-4bea-8f84-84098720361a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c945ff72665481b9c9506a3cca6cfde'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:49.253371 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-de44d7b7-d1ba-45a0-9469-bacc464837a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c9ea9542e1b4d95b9af64192e8544f1', 'request_id': 'req-de44d7b7-d1ba-45a0-9469-bacc464837a3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:49.340147 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-128129e2-12e0-4bea-8f84-84098720361a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c945ff72665481b9c9506a3cca6cfde'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:49.353426 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-128129e2-12e0-4bea-8f84-84098720361a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c945ff72665481b9c9506a3cca6cfde'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:49.354019 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-128129e2-12e0-4bea-8f84-84098720361a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c945ff72665481b9c9506a3cca6cfde'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:49.354757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-128129e2-12e0-4bea-8f84-84098720361a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c945ff72665481b9c9506a3cca6cfde'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:49.370968 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1345/2708] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:22:48 2022] DELETE /identity/v3/users/9c945ff72665481b9c9506a3cca6cfde => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:49.375174 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433fc050-67e5-46f1-9723-def27f4cde72 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:49.375467 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433fc050-67e5-46f1-9723-def27f4cde72 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:49.375751 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433fc050-67e5-46f1-9723-def27f4cde72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:49.396168 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-433fc050-67e5-46f1-9723-def27f4cde72 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:49.397388 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-433fc050-67e5-46f1-9723-def27f4cde72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:49.403406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-de44d7b7-d1ba-45a0-9469-bacc464837a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c9ea9542e1b4d95b9af64192e8544f1', 'request_id': 'req-de44d7b7-d1ba-45a0-9469-bacc464837a3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:49.415956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-de44d7b7-d1ba-45a0-9469-bacc464837a3 admin admin] The token cache is being invalidate because project 2c9ea9542e1b4d95b9af64192e8544f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:22:49.416245 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-de44d7b7-d1ba-45a0-9469-bacc464837a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:49.417065 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-de44d7b7-d1ba-45a0-9469-bacc464837a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:49.418916 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1364/2709] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:22:48 2022] DELETE /identity/v3/projects/2c9ea9542e1b4d95b9af64192e8544f1 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:49.423701 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c79aa6-d569-46aa-b456-85486663ad58 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:49.423989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c79aa6-d569-46aa-b456-85486663ad58 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:49.424268 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c79aa6-d569-46aa-b456-85486663ad58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:49.446079 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-74c79aa6-d569-46aa-b456-85486663ad58 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:49.451170 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-74c79aa6-d569-46aa-b456-85486663ad58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:49.464108 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:49.479059 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1346/2710] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:22:49.493967 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433fc050-67e5-46f1-9723-def27f4cde72 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:49.495749 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-433fc050-67e5-46f1-9723-def27f4cde72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:49.527343 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1365/2711] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:22:49.535204 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbbb1f-2de1-4ada-b6c3-7a2a65009f71 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:49.537056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbbb1f-2de1-4ada-b6c3-7a2a65009f71 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:49.537336 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbbb1f-2de1-4ada-b6c3-7a2a65009f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:49.560126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-52cbbb1f-2de1-4ada-b6c3-7a2a65009f71 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:49.561384 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-52cbbb1f-2de1-4ada-b6c3-7a2a65009f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:49.561911 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433fc050-67e5-46f1-9723-def27f4cde72 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:49.565592 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a37dcae-4da2-4f1c-9914-408a702654a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:49.566196 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37dcae-4da2-4f1c-9914-408a702654a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:49.567057 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37dcae-4da2-4f1c-9914-408a702654a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:49.567368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37dcae-4da2-4f1c-9914-408a702654a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:49.573268 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1366/2712] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:49.574433 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbbb1f-2de1-4ada-b6c3-7a2a65009f71 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:49.577434 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a37dcae-4da2-4f1c-9914-408a702654a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:49.578998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a37dcae-4da2-4f1c-9914-408a702654a0 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:49.584112 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbbb1f-2de1-4ada-b6c3-7a2a65009f71 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:49.588730 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f911e3-756f-4ed3-b629-88e85900f0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:49.589326 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f911e3-756f-4ed3-b629-88e85900f0d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:49.594771 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f911e3-756f-4ed3-b629-88e85900f0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:49.594771 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f911e3-756f-4ed3-b629-88e85900f0d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:49.595577 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1347/2713] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:22:49.595687 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f911e3-756f-4ed3-b629-88e85900f0d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:22:49.595687 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f911e3-756f-4ed3-b629-88e85900f0d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:49.595687 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f911e3-756f-4ed3-b629-88e85900f0d4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:49.608276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1367/2714] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 17:22:49.609197 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a37dcae-4da2-4f1c-9914-408a702654a0 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:49.620184 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a37dcae-4da2-4f1c-9914-408a702654a0 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:49.625181 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52c8da-c274-4c74-b8a0-24c2d6998243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:49.625884 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52c8da-c274-4c74-b8a0-24c2d6998243 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:49.626219 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52c8da-c274-4c74-b8a0-24c2d6998243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:49.628609 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52c8da-c274-4c74-b8a0-24c2d6998243 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:49.628609 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52c8da-c274-4c74-b8a0-24c2d6998243 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:49.630111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52c8da-c274-4c74-b8a0-24c2d6998243 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:49.742425 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1348/2715] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:22:49 2022] POST /identity/v3/projects => generated 363 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:22:49.746694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f911e3-756f-4ed3-b629-88e85900f0d4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:49.757260 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f911e3-756f-4ed3-b629-88e85900f0d4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:49.821453 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f0e74a-5558-4350-8824-30a6555ee5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:49.822061 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0e74a-5558-4350-8824-30a6555ee5ec admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:49.822342 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0e74a-5558-4350-8824-30a6555ee5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:49.822641 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0e74a-5558-4350-8824-30a6555ee5ec admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:49.823327 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f0e74a-5558-4350-8824-30a6555ee5ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:49.825310 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f0e74a-5558-4350-8824-30a6555ee5ec admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:49.836404 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:49.923463 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1368/2716] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:22:49 2022] POST /identity/v3/users => generated 282 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:22:49.928735 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52c8da-c274-4c74-b8a0-24c2d6998243 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:49.940845 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52c8da-c274-4c74-b8a0-24c2d6998243 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:49.948187 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ea370-e194-4ce9-b0b3-9e0037761921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:49.948187 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ea370-e194-4ce9-b0b3-9e0037761921 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:49.948484 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ea370-e194-4ce9-b0b3-9e0037761921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:49.948786 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ea370-e194-4ce9-b0b3-9e0037761921 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:49.949624 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ea370-e194-4ce9-b0b3-9e0037761921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:49.951427 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ea370-e194-4ce9-b0b3-9e0037761921 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:49.963533 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1349/2717] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:49.967747 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f0e74a-5558-4350-8824-30a6555ee5ec admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:49.979505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f0e74a-5558-4350-8824-30a6555ee5ec admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:49.984572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777bd13b-9db0-4648-89ce-110faad7aeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:49.985202 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777bd13b-9db0-4648-89ce-110faad7aeb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:49.985483 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777bd13b-9db0-4648-89ce-110faad7aeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:49.985777 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777bd13b-9db0-4648-89ce-110faad7aeb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:49.986459 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777bd13b-9db0-4648-89ce-110faad7aeb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:49.988073 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777bd13b-9db0-4648-89ce-110faad7aeb3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:49.999406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1369/2718] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:50.005229 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ea370-e194-4ce9-b0b3-9e0037761921 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:50.017363 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ea370-e194-4ce9-b0b3-9e0037761921 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:50.022867 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfddc309-e55b-4414-8288-3aee1ad4d9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:50.023686 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddc309-e55b-4414-8288-3aee1ad4d9ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:50.023998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddc309-e55b-4414-8288-3aee1ad4d9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:50.024267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddc309-e55b-4414-8288-3aee1ad4d9ce admin admin] PATH_INFO: `/v3/projects/6d77910ea6a6498da86d470e54ce4fb8/users/b8c54e4edff24f75b28febf51f772672/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:50.040462 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022d572-e23b-46d6-a25f-6d0245e0dea4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:50.040817 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022d572-e23b-46d6-a25f-6d0245e0dea4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:50.041209 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfddc309-e55b-4414-8288-3aee1ad4d9ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d77910ea6a6498da86d470e54ce4fb8, user_id=b8c54e4edff24f75b28febf51f772672, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:50.041385 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022d572-e23b-46d6-a25f-6d0245e0dea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:50.043862 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfddc309-e55b-4414-8288-3aee1ad4d9ce admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:50.070702 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c022d572-e23b-46d6-a25f-6d0245e0dea4 None None] MFA Rules not processed for user `59800960b5a44ca78d5cb67fee10d366`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:50.071946 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c022d572-e23b-46d6-a25f-6d0245e0dea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:50.152063 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1370/2719] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:22:50.153798 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1350/2720] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:50 2022] PUT /identity/v3/projects/6d77910ea6a6498da86d470e54ce4fb8/users/b8c54e4edff24f75b28febf51f772672/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:50.158611 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c022d572-e23b-46d6-a25f-6d0245e0dea4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:50.162941 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c022d572-e23b-46d6-a25f-6d0245e0dea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:50.163519 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfddc309-e55b-4414-8288-3aee1ad4d9ce admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:50.165264 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cfddc309-e55b-4414-8288-3aee1ad4d9ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:50.177492 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c022d572-e23b-46d6-a25f-6d0245e0dea4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:50.224317 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfddc309-e55b-4414-8288-3aee1ad4d9ce admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:50.227965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25f231-0e18-43a6-bce9-47dee9d3f8ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:50.228566 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25f231-0e18-43a6-bce9-47dee9d3f8ea service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:50.228836 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25f231-0e18-43a6-bce9-47dee9d3f8ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:50.229099 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25f231-0e18-43a6-bce9-47dee9d3f8ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:50.230817 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0a25f231-0e18-43a6-bce9-47dee9d3f8ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:50.236215 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9d587-ef8c-43c9-903d-6c6d05f0c48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:50.236913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9d587-ef8c-43c9-903d-6c6d05f0c48b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:50.237191 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9d587-ef8c-43c9-903d-6c6d05f0c48b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:50.237484 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9d587-ef8c-43c9-903d-6c6d05f0c48b admin admin] PATH_INFO: `/v3/projects/41b17d07b4e044faa078e6001fd9b49c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:50.239258 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9d587-ef8c-43c9-903d-6c6d05f0c48b admin admin] RBAC: Authorizing `identity:delete_project(project_id=41b17d07b4e044faa078e6001fd9b49c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:50.241396 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9d587-ef8c-43c9-903d-6c6d05f0c48b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:50.290056 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:50.293678 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25f231-0e18-43a6-bce9-47dee9d3f8ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:50.295120 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25f231-0e18-43a6-bce9-47dee9d3f8ea service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:50.304779 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1351/2721] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:22:50.306226 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac3027-df65-49dd-ba8b-58d20e731469 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:50.306649 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac3027-df65-49dd-ba8b-58d20e731469 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:50.307072 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac3027-df65-49dd-ba8b-58d20e731469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:50.336479 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-17ac3027-df65-49dd-ba8b-58d20e731469 None None] MFA Rules not processed for user `b8c54e4edff24f75b28febf51f772672`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:50.337861 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-17ac3027-df65-49dd-ba8b-58d20e731469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:50.404399 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4da9d587-ef8c-43c9-903d-6c6d05f0c48b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41b17d07b4e044faa078e6001fd9b49c', 'request_id': 'req-4da9d587-ef8c-43c9-903d-6c6d05f0c48b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:50.411612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1352/2722] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:50.412989 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4da9d587-ef8c-43c9-903d-6c6d05f0c48b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41b17d07b4e044faa078e6001fd9b49c', 'request_id': 'req-4da9d587-ef8c-43c9-903d-6c6d05f0c48b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:50.419063 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac3027-df65-49dd-ba8b-58d20e731469 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:50.420464 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-17ac3027-df65-49dd-ba8b-58d20e731469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:50.432822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac3027-df65-49dd-ba8b-58d20e731469 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:50.484432 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ce3cc-b180-421f-910b-8d4413144486 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:50.485043 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ce3cc-b180-421f-910b-8d4413144486 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:50.485318 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ce3cc-b180-421f-910b-8d4413144486 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:50.485588 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ce3cc-b180-421f-910b-8d4413144486 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:50.547343 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ce3cc-b180-421f-910b-8d4413144486 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:50.548756 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ce3cc-b180-421f-910b-8d4413144486 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:50.558242 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1353/2723] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:22:50.580348 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4da9d587-ef8c-43c9-903d-6c6d05f0c48b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41b17d07b4e044faa078e6001fd9b49c', 'request_id': 'req-4da9d587-ef8c-43c9-903d-6c6d05f0c48b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:50.593274 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4da9d587-ef8c-43c9-903d-6c6d05f0c48b admin admin] The token cache is being invalidate because project 41b17d07b4e044faa078e6001fd9b49c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:22:50.594923 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4da9d587-ef8c-43c9-903d-6c6d05f0c48b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:50.597086 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4da9d587-ef8c-43c9-903d-6c6d05f0c48b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:50.599494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1371/2724] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:22:50 2022] DELETE /identity/v3/projects/41b17d07b4e044faa078e6001fd9b49c => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:50.608490 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ce3cc-b180-421f-910b-8d4413144486 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:50.671549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ce3cc-b180-421f-910b-8d4413144486 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:50.676929 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b9e6a-d552-41fc-854e-a319d0765f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:50.678263 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b9e6a-d552-41fc-854e-a319d0765f09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:50.678574 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b9e6a-d552-41fc-854e-a319d0765f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:50.678848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b9e6a-d552-41fc-854e-a319d0765f09 admin admin] PATH_INFO: `/v3/users/4662b07b5bf04268828f84412ac6cbc7` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:50.679993 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b9e6a-d552-41fc-854e-a319d0765f09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4662b07b5bf04268828f84412ac6cbc7)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:50.681889 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b9e6a-d552-41fc-854e-a319d0765f09 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:51.034733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a89b9e6a-d552-41fc-854e-a319d0765f09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4662b07b5bf04268828f84412ac6cbc7'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:51.180554 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a89b9e6a-d552-41fc-854e-a319d0765f09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4662b07b5bf04268828f84412ac6cbc7'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:51.194278 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a89b9e6a-d552-41fc-854e-a319d0765f09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4662b07b5bf04268828f84412ac6cbc7'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:51.195425 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a89b9e6a-d552-41fc-854e-a319d0765f09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4662b07b5bf04268828f84412ac6cbc7'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:51.196102 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a89b9e6a-d552-41fc-854e-a319d0765f09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4662b07b5bf04268828f84412ac6cbc7'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:51.211736 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1354/2725] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:22:50 2022] DELETE /identity/v3/users/4662b07b5bf04268828f84412ac6cbc7 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:51.289767 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:51.380793 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:51.690370 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9d587-ef8c-43c9-903d-6c6d05f0c48b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:51.693086 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4da9d587-ef8c-43c9-903d-6c6d05f0c48b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:51.706392 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9d587-ef8c-43c9-903d-6c6d05f0c48b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:51.765299 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d0b5d-f1b4-47e5-83e7-a7b1056b3d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:51.765851 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d0b5d-f1b4-47e5-83e7-a7b1056b3d66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:51.766127 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d0b5d-f1b4-47e5-83e7-a7b1056b3d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:51.766427 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d0b5d-f1b4-47e5-83e7-a7b1056b3d66 admin admin] PATH_INFO: `/v3/projects/0e610a95c6044556b7a7aa71f567b1f2` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:51.767663 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72d0b5d-f1b4-47e5-83e7-a7b1056b3d66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e610a95c6044556b7a7aa71f567b1f2)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:51.769793 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72d0b5d-f1b4-47e5-83e7-a7b1056b3d66 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:51.934170 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a72d0b5d-f1b4-47e5-83e7-a7b1056b3d66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e610a95c6044556b7a7aa71f567b1f2', 'request_id': 'req-a72d0b5d-f1b4-47e5-83e7-a7b1056b3d66'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:51.940243 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a72d0b5d-f1b4-47e5-83e7-a7b1056b3d66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e610a95c6044556b7a7aa71f567b1f2', 'request_id': 'req-a72d0b5d-f1b4-47e5-83e7-a7b1056b3d66'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:52.065930 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a72d0b5d-f1b4-47e5-83e7-a7b1056b3d66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e610a95c6044556b7a7aa71f567b1f2', 'request_id': 'req-a72d0b5d-f1b4-47e5-83e7-a7b1056b3d66'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:52.076945 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a72d0b5d-f1b4-47e5-83e7-a7b1056b3d66 admin admin] The token cache is being invalidate because project 0e610a95c6044556b7a7aa71f567b1f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:22:52.077231 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a72d0b5d-f1b4-47e5-83e7-a7b1056b3d66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:52.077869 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a72d0b5d-f1b4-47e5-83e7-a7b1056b3d66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:52.079395 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1372/2726] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:22:51 2022] DELETE /identity/v3/projects/0e610a95c6044556b7a7aa71f567b1f2 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:52.085669 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8a956-6bf1-4ba1-88ef-3113e90e4392 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:52.086162 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8a956-6bf1-4ba1-88ef-3113e90e4392 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:52.086266 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8a956-6bf1-4ba1-88ef-3113e90e4392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:52.106111 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e0b8a956-6bf1-4ba1-88ef-3113e90e4392 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:52.107357 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e0b8a956-6bf1-4ba1-88ef-3113e90e4392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:52.167300 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1355/2727] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:52.175280 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e46da-292c-4c24-9aff-254621672b08 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:52.175564 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e46da-292c-4c24-9aff-254621672b08 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:52.175834 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e46da-292c-4c24-9aff-254621672b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:52.196311 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-157e46da-292c-4c24-9aff-254621672b08 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:52.197506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-157e46da-292c-4c24-9aff-254621672b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:52.205306 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1373/2728] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:52.209662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8a956-6bf1-4ba1-88ef-3113e90e4392 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:52.219348 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8a956-6bf1-4ba1-88ef-3113e90e4392 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:52.223851 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af4c0c-b1c3-48b6-9b4f-670bd914a3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:52.224432 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af4c0c-b1c3-48b6-9b4f-670bd914a3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:52.224701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af4c0c-b1c3-48b6-9b4f-670bd914a3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:52.224963 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af4c0c-b1c3-48b6-9b4f-670bd914a3f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:52.226108 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af4c0c-b1c3-48b6-9b4f-670bd914a3f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:22:52.226437 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af4c0c-b1c3-48b6-9b4f-670bd914a3f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:52.228025 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af4c0c-b1c3-48b6-9b4f-670bd914a3f2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:52.239617 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1356/2729] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:22:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:22:52.244964 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e46da-292c-4c24-9aff-254621672b08 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:52.254467 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e46da-292c-4c24-9aff-254621672b08 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:52.259550 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1772e53c-01fe-4293-8f41-b2831736f9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:52.260144 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772e53c-01fe-4293-8f41-b2831736f9d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:52.260423 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772e53c-01fe-4293-8f41-b2831736f9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:52.260695 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772e53c-01fe-4293-8f41-b2831736f9d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:52.261322 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1772e53c-01fe-4293-8f41-b2831736f9d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:52.263232 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1772e53c-01fe-4293-8f41-b2831736f9d9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:52.377969 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1374/2730] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:22:52 2022] POST /identity/v3/projects => generated 375 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:22:52.383409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af4c0c-b1c3-48b6-9b4f-670bd914a3f2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:52.391439 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1772e53c-01fe-4293-8f41-b2831736f9d9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:52.393182 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1772e53c-01fe-4293-8f41-b2831736f9d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:52.393700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af4c0c-b1c3-48b6-9b4f-670bd914a3f2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:52.438588 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:52.451681 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1772e53c-01fe-4293-8f41-b2831736f9d9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:52.453560 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e319c9-7054-41de-824c-beaec74c2235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:52.454147 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e319c9-7054-41de-824c-beaec74c2235 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:52.454465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e319c9-7054-41de-824c-beaec74c2235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:52.454741 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e319c9-7054-41de-824c-beaec74c2235 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:52.455405 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e319c9-7054-41de-824c-beaec74c2235 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:52.460234 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af480112-4e22-49ee-9c34-0c063a2acb8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:52.460234 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af480112-4e22-49ee-9c34-0c063a2acb8e service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:52.460234 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af480112-4e22-49ee-9c34-0c063a2acb8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:52.460234 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af480112-4e22-49ee-9c34-0c063a2acb8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:52.460628 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e319c9-7054-41de-824c-beaec74c2235 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:52.460722 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-af480112-4e22-49ee-9c34-0c063a2acb8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:52.473060 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:52.529106 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af480112-4e22-49ee-9c34-0c063a2acb8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:52.530576 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af480112-4e22-49ee-9c34-0c063a2acb8e service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:52.541406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1375/2731] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:52 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:22:52.546109 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1357/2732] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:22:52 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:22:52.546707 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af480112-4e22-49ee-9c34-0c063a2acb8e service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:52.558079 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af480112-4e22-49ee-9c34-0c063a2acb8e service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:52.564688 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0247fb1-2c4b-490f-8eac-07bfbdb56e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:52.565483 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0247fb1-2c4b-490f-8eac-07bfbdb56e5a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:52.566834 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0247fb1-2c4b-490f-8eac-07bfbdb56e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:52.566950 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0247fb1-2c4b-490f-8eac-07bfbdb56e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:52.567678 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0247fb1-2c4b-490f-8eac-07bfbdb56e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:52.569381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0247fb1-2c4b-490f-8eac-07bfbdb56e5a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:52.583610 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1376/2733] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:52.588726 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e319c9-7054-41de-824c-beaec74c2235 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:52.599616 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e319c9-7054-41de-824c-beaec74c2235 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:52.606597 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22fd96b-33d7-4b1c-bec3-5c43e9340e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:52.607215 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fd96b-33d7-4b1c-bec3-5c43e9340e6c admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:52.607486 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fd96b-33d7-4b1c-bec3-5c43e9340e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:52.607749 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fd96b-33d7-4b1c-bec3-5c43e9340e6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:52.608460 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22fd96b-33d7-4b1c-bec3-5c43e9340e6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:52.610137 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22fd96b-33d7-4b1c-bec3-5c43e9340e6c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:52.622002 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1358/2734] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:52.626566 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0247fb1-2c4b-490f-8eac-07bfbdb56e5a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:52.637566 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0247fb1-2c4b-490f-8eac-07bfbdb56e5a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:52.642369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44129f06-2fc3-4e50-8191-b5ffbf078ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:52.643037 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44129f06-2fc3-4e50-8191-b5ffbf078ea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:52.643319 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44129f06-2fc3-4e50-8191-b5ffbf078ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:52.643590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44129f06-2fc3-4e50-8191-b5ffbf078ea2 admin admin] PATH_INFO: `/v3/projects/871f1643f87f4a5bafa92f4f94eee37f/users/b13324aa6f10468689624f382aae6724/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:52.659398 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44129f06-2fc3-4e50-8191-b5ffbf078ea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=871f1643f87f4a5bafa92f4f94eee37f, user_id=b13324aa6f10468689624f382aae6724, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:52.662039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44129f06-2fc3-4e50-8191-b5ffbf078ea2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:52.792172 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1377/2735] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:52 2022] PUT /identity/v3/projects/871f1643f87f4a5bafa92f4f94eee37f/users/b13324aa6f10468689624f382aae6724/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:52.803681 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649970aa-d321-4d6e-9db7-da74946e90fa None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:52.803963 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649970aa-d321-4d6e-9db7-da74946e90fa None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:52.804234 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649970aa-d321-4d6e-9db7-da74946e90fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:52.833199 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-649970aa-d321-4d6e-9db7-da74946e90fa None None] MFA Rules not processed for user `b13324aa6f10468689624f382aae6724`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:52.834345 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-649970aa-d321-4d6e-9db7-da74946e90fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:52.903703 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1359/2736] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:52.908203 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44129f06-2fc3-4e50-8191-b5ffbf078ea2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:52.969069 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44129f06-2fc3-4e50-8191-b5ffbf078ea2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:52.973825 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a1b2b-9590-472f-9cca-ea1199425a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:52.974446 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a1b2b-9590-472f-9cca-ea1199425a06 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:52.974735 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a1b2b-9590-472f-9cca-ea1199425a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:52.975043 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a1b2b-9590-472f-9cca-ea1199425a06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:52.975749 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a1b2b-9590-472f-9cca-ea1199425a06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:52.977706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a1b2b-9590-472f-9cca-ea1199425a06 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:53.137216 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1378/2737] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:22:52 2022] POST /identity/v3/projects => generated 375 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:22:53.141090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649970aa-d321-4d6e-9db7-da74946e90fa None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:53.151901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649970aa-d321-4d6e-9db7-da74946e90fa None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:53.207581 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996a82c-ef79-4bb3-b6e6-fb7ed3fa42dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:53.208178 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996a82c-ef79-4bb3-b6e6-fb7ed3fa42dd admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:53.208449 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996a82c-ef79-4bb3-b6e6-fb7ed3fa42dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:53.208716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996a82c-ef79-4bb3-b6e6-fb7ed3fa42dd admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:53.209358 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1996a82c-ef79-4bb3-b6e6-fb7ed3fa42dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:53.211609 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1996a82c-ef79-4bb3-b6e6-fb7ed3fa42dd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:53.361659 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1360/2738] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:22:53 2022] POST /identity/v3/users => generated 294 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:22:53.362078 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a1b2b-9590-472f-9cca-ea1199425a06 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:53.373556 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a1b2b-9590-472f-9cca-ea1199425a06 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:53.378531 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d88dc6-e62c-411d-9f7f-14638d60300b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:53.379325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d88dc6-e62c-411d-9f7f-14638d60300b admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:53.379465 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d88dc6-e62c-411d-9f7f-14638d60300b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:53.379738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d88dc6-e62c-411d-9f7f-14638d60300b admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:53.380530 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d88dc6-e62c-411d-9f7f-14638d60300b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:53.383050 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d88dc6-e62c-411d-9f7f-14638d60300b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:53.396072 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1379/2739] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:53.404878 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996a82c-ef79-4bb3-b6e6-fb7ed3fa42dd admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:53.415647 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996a82c-ef79-4bb3-b6e6-fb7ed3fa42dd admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:53.420539 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335948f-35b2-48a9-85b6-0525d8afa722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:53.421180 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335948f-35b2-48a9-85b6-0525d8afa722 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:53.424870 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335948f-35b2-48a9-85b6-0525d8afa722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:53.424870 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335948f-35b2-48a9-85b6-0525d8afa722 admin admin] PATH_INFO: `/v3/projects/e09ff979dcf045a190e1534a265783b1/users/b64824e848694cd6a430d39117991003/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:53.437038 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9335948f-35b2-48a9-85b6-0525d8afa722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e09ff979dcf045a190e1534a265783b1, user_id=b64824e848694cd6a430d39117991003, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:53.439714 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9335948f-35b2-48a9-85b6-0525d8afa722 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:53.558643 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1361/2740] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:53 2022] PUT /identity/v3/projects/e09ff979dcf045a190e1534a265783b1/users/b64824e848694cd6a430d39117991003/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:53.562896 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d88dc6-e62c-411d-9f7f-14638d60300b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:53.572946 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d88dc6-e62c-411d-9f7f-14638d60300b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:53.629571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7dff7-2eb0-4d64-a4aa-cef6e0877ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:53.630147 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7dff7-2eb0-4d64-a4aa-cef6e0877ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:53.630452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7dff7-2eb0-4d64-a4aa-cef6e0877ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:53.630739 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7dff7-2eb0-4d64-a4aa-cef6e0877ff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:53.631401 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7dff7-2eb0-4d64-a4aa-cef6e0877ff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:53.633266 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7dff7-2eb0-4d64-a4aa-cef6e0877ff9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:53.643196 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1380/2741] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:22:53 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:22:53.647388 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335948f-35b2-48a9-85b6-0525d8afa722 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:53.660796 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335948f-35b2-48a9-85b6-0525d8afa722 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:53.672803 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47abe40-d2ec-4648-bd80-e18a7cf75416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:53.673851 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47abe40-d2ec-4648-bd80-e18a7cf75416 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:53.673851 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47abe40-d2ec-4648-bd80-e18a7cf75416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:53.674633 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47abe40-d2ec-4648-bd80-e18a7cf75416 admin admin] PATH_INFO: `/v3/projects/e09ff979dcf045a190e1534a265783b1/users/b64824e848694cd6a430d39117991003/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:53.677049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47abe40-d2ec-4648-bd80-e18a7cf75416 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e09ff979dcf045a190e1534a265783b1, user_id=b64824e848694cd6a430d39117991003, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:53.678216 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:53.680568 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47abe40-d2ec-4648-bd80-e18a7cf75416 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:53.691867 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1362/2742] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:22:53 2022] PUT /identity/v3/projects/e09ff979dcf045a190e1534a265783b1/users/b64824e848694cd6a430d39117991003/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:53.703544 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa6257-908b-4d95-b3da-f05f3f42c358 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:53.703831 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa6257-908b-4d95-b3da-f05f3f42c358 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:53.704107 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa6257-908b-4d95-b3da-f05f3f42c358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:53.734293 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3baa6257-908b-4d95-b3da-f05f3f42c358 None None] MFA Rules not processed for user `b64824e848694cd6a430d39117991003`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:53.735738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3baa6257-908b-4d95-b3da-f05f3f42c358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:53.809777 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1381/2743] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:53.822071 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47abe40-d2ec-4648-bd80-e18a7cf75416 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:53.823837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e47abe40-d2ec-4648-bd80-e18a7cf75416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:53.830305 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:53.882389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47abe40-d2ec-4648-bd80-e18a7cf75416 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:53.886200 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f5fd4-0982-413c-b586-aaf4a72691cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:53.886830 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f5fd4-0982-413c-b586-aaf4a72691cb service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:53.887102 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f5fd4-0982-413c-b586-aaf4a72691cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:53.887369 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f5fd4-0982-413c-b586-aaf4a72691cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:53.943280 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705f5fd4-0982-413c-b586-aaf4a72691cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:53.945367 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705f5fd4-0982-413c-b586-aaf4a72691cb service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:53.954191 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1363/2744] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:22:57.391914 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa6257-908b-4d95-b3da-f05f3f42c358 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:57.459025 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa6257-908b-4d95-b3da-f05f3f42c358 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:57.464586 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827562b-fcbb-44ba-9641-336993f41633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:57.465253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827562b-fcbb-44ba-9641-336993f41633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:57.465530 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827562b-fcbb-44ba-9641-336993f41633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:57.465799 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827562b-fcbb-44ba-9641-336993f41633 admin admin] PATH_INFO: `/v3/users/b13324aa6f10468689624f382aae6724` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:57.466999 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827562b-fcbb-44ba-9641-336993f41633 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b13324aa6f10468689624f382aae6724)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:57.468946 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827562b-fcbb-44ba-9641-336993f41633 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:57.728423 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b827562b-fcbb-44ba-9641-336993f41633 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b13324aa6f10468689624f382aae6724'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:57.814505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b827562b-fcbb-44ba-9641-336993f41633 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b13324aa6f10468689624f382aae6724'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:57.822177 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b827562b-fcbb-44ba-9641-336993f41633 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b13324aa6f10468689624f382aae6724'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:57.830694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:57.836134 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b827562b-fcbb-44ba-9641-336993f41633 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b13324aa6f10468689624f382aae6724'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:57.837424 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b827562b-fcbb-44ba-9641-336993f41633 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b13324aa6f10468689624f382aae6724'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:57.843838 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1382/2745] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:22:57 2022] DELETE /identity/v3/users/b13324aa6f10468689624f382aae6724 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:57.848412 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d021388-0ba3-4ea3-968e-42c760225f0b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:57.848878 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d021388-0ba3-4ea3-968e-42c760225f0b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:57.849008 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d021388-0ba3-4ea3-968e-42c760225f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:57.868991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1d021388-0ba3-4ea3-968e-42c760225f0b None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:22:57.870203 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1d021388-0ba3-4ea3-968e-42c760225f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:57.931872 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1364/2746] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:22:57.941823 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827562b-fcbb-44ba-9641-336993f41633 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:57.943241 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b827562b-fcbb-44ba-9641-336993f41633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:58.000596 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827562b-fcbb-44ba-9641-336993f41633 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:58.005914 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8cff1a-cc5d-45a0-9622-622366faa34a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:58.007108 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cff1a-cc5d-45a0-9622-622366faa34a service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:58.007452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cff1a-cc5d-45a0-9622-622366faa34a service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:58.007834 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cff1a-cc5d-45a0-9622-622366faa34a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:58.022850 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8cff1a-cc5d-45a0-9622-622366faa34a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:58.022850 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8cff1a-cc5d-45a0-9622-622366faa34a service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:58.035934 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1383/2747] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:22:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:22:58.473722 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d021388-0ba3-4ea3-968e-42c760225f0b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:58.474949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1d021388-0ba3-4ea3-968e-42c760225f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:58.490128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d021388-0ba3-4ea3-968e-42c760225f0b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:58.496431 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0a693-4e12-483a-8971-2467f7835ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:58.497333 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0a693-4e12-483a-8971-2467f7835ae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:58.497804 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0a693-4e12-483a-8971-2467f7835ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:58.498150 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0a693-4e12-483a-8971-2467f7835ae1 admin admin] PATH_INFO: `/v3/projects/871f1643f87f4a5bafa92f4f94eee37f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:58.500439 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af0a693-4e12-483a-8971-2467f7835ae1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=871f1643f87f4a5bafa92f4f94eee37f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:58.503619 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af0a693-4e12-483a-8971-2467f7835ae1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:58.619195 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9af0a693-4e12-483a-8971-2467f7835ae1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '871f1643f87f4a5bafa92f4f94eee37f', 'request_id': 'req-9af0a693-4e12-483a-8971-2467f7835ae1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:58.622918 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:22:58.711462 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9af0a693-4e12-483a-8971-2467f7835ae1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '871f1643f87f4a5bafa92f4f94eee37f', 'request_id': 'req-9af0a693-4e12-483a-8971-2467f7835ae1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:58.718546 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9af0a693-4e12-483a-8971-2467f7835ae1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '871f1643f87f4a5bafa92f4f94eee37f', 'request_id': 'req-9af0a693-4e12-483a-8971-2467f7835ae1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:58.729661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9af0a693-4e12-483a-8971-2467f7835ae1 admin admin] The token cache is being invalidate because project 871f1643f87f4a5bafa92f4f94eee37f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:22:58.729950 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9af0a693-4e12-483a-8971-2467f7835ae1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:58.730623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9af0a693-4e12-483a-8971-2467f7835ae1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:58.732109 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1365/2748] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:22:58 2022] DELETE /identity/v3/projects/871f1643f87f4a5bafa92f4f94eee37f => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:58.736185 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8cff1a-cc5d-45a0-9622-622366faa34a service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:58.746591 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8cff1a-cc5d-45a0-9622-622366faa34a service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:58.802456 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada76f9-1082-496c-8957-61885160a83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:58.803146 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada76f9-1082-496c-8957-61885160a83b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:58.803457 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada76f9-1082-496c-8957-61885160a83b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:58.803734 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada76f9-1082-496c-8957-61885160a83b admin admin] PATH_INFO: `/v3/users/b64824e848694cd6a430d39117991003` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:58.804848 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada76f9-1082-496c-8957-61885160a83b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b64824e848694cd6a430d39117991003)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:58.807668 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada76f9-1082-496c-8957-61885160a83b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:59.016982 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1ada76f9-1082-496c-8957-61885160a83b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b64824e848694cd6a430d39117991003'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:59.136083 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1ada76f9-1082-496c-8957-61885160a83b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b64824e848694cd6a430d39117991003'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:59.146230 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1ada76f9-1082-496c-8957-61885160a83b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b64824e848694cd6a430d39117991003'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:59.158030 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1ada76f9-1082-496c-8957-61885160a83b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b64824e848694cd6a430d39117991003'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:59.158655 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1ada76f9-1082-496c-8957-61885160a83b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b64824e848694cd6a430d39117991003'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:22:59.164763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1384/2749] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:22:58 2022] DELETE /identity/v3/users/b64824e848694cd6a430d39117991003 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:22:59.673741 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0a693-4e12-483a-8971-2467f7835ae1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:22:59.675221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9af0a693-4e12-483a-8971-2467f7835ae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:22:59.688924 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0a693-4e12-483a-8971-2467f7835ae1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:22:59.747781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24adbc9f-60e5-43ff-9f40-4c006e306509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:22:59.748432 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24adbc9f-60e5-43ff-9f40-4c006e306509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:22:59.748703 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24adbc9f-60e5-43ff-9f40-4c006e306509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:22:59.748970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24adbc9f-60e5-43ff-9f40-4c006e306509 admin admin] PATH_INFO: `/v3/projects/e09ff979dcf045a190e1534a265783b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:22:59.750112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24adbc9f-60e5-43ff-9f40-4c006e306509 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e09ff979dcf045a190e1534a265783b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:22:59.752053 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24adbc9f-60e5-43ff-9f40-4c006e306509 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:22:59.889026 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-24adbc9f-60e5-43ff-9f40-4c006e306509 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e09ff979dcf045a190e1534a265783b1', 'request_id': 'req-24adbc9f-60e5-43ff-9f40-4c006e306509'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:00.026407 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-24adbc9f-60e5-43ff-9f40-4c006e306509 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e09ff979dcf045a190e1534a265783b1', 'request_id': 'req-24adbc9f-60e5-43ff-9f40-4c006e306509'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:00.033379 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-24adbc9f-60e5-43ff-9f40-4c006e306509 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e09ff979dcf045a190e1534a265783b1', 'request_id': 'req-24adbc9f-60e5-43ff-9f40-4c006e306509'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:00.045155 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-24adbc9f-60e5-43ff-9f40-4c006e306509 admin admin] The token cache is being invalidate because project e09ff979dcf045a190e1534a265783b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:23:00.045441 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-24adbc9f-60e5-43ff-9f40-4c006e306509 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:00.046082 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-24adbc9f-60e5-43ff-9f40-4c006e306509 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:00.047588 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1366/2750] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:22:59 2022] DELETE /identity/v3/projects/e09ff979dcf045a190e1534a265783b1 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:23:00.053680 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e192b-9255-4d6e-aa83-8538386ae40c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:00.054152 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e192b-9255-4d6e-aa83-8538386ae40c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:00.054270 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e192b-9255-4d6e-aa83-8538386ae40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:00.074158 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-021e192b-9255-4d6e-aa83-8538386ae40c None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:23:00.075397 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-021e192b-9255-4d6e-aa83-8538386ae40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:00.080738 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:23:00.135247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1385/2751] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:23:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:23:00.143424 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed4ef4-e4a8-4319-ae17-98fe40912af3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:00.143675 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed4ef4-e4a8-4319-ae17-98fe40912af3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:00.143949 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed4ef4-e4a8-4319-ae17-98fe40912af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:00.164430 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-cfed4ef4-e4a8-4319-ae17-98fe40912af3 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:23:00.165702 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cfed4ef4-e4a8-4319-ae17-98fe40912af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:00.174164 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1367/2752] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:23:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:23:00.178296 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e192b-9255-4d6e-aa83-8538386ae40c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:00.188574 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e192b-9255-4d6e-aa83-8538386ae40c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:00.193217 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4023ad-7538-4cbc-b2cd-403adbbe2e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:00.193857 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4023ad-7538-4cbc-b2cd-403adbbe2e54 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:00.194131 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4023ad-7538-4cbc-b2cd-403adbbe2e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:00.194464 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4023ad-7538-4cbc-b2cd-403adbbe2e54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:00.195107 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4023ad-7538-4cbc-b2cd-403adbbe2e54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:23:00.195403 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4023ad-7538-4cbc-b2cd-403adbbe2e54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:00.197613 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4023ad-7538-4cbc-b2cd-403adbbe2e54 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:00.209101 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1386/2753] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:23:00.214306 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed4ef4-e4a8-4319-ae17-98fe40912af3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:00.223759 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed4ef4-e4a8-4319-ae17-98fe40912af3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:00.228781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e2e45-c527-44ca-8b2f-9f2e22151143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:00.229370 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e2e45-c527-44ca-8b2f-9f2e22151143 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:00.229662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e2e45-c527-44ca-8b2f-9f2e22151143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:00.229926 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e2e45-c527-44ca-8b2f-9f2e22151143 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:00.230629 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e2e45-c527-44ca-8b2f-9f2e22151143 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:00.232509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e2e45-c527-44ca-8b2f-9f2e22151143 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:00.336210 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1368/2754] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:23:00 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:23:00.341109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4023ad-7538-4cbc-b2cd-403adbbe2e54 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:00.351848 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4023ad-7538-4cbc-b2cd-403adbbe2e54 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:00.412224 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c047b2-8e2f-4626-ad49-643e5db979d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:00.412763 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c047b2-8e2f-4626-ad49-643e5db979d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:00.413038 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c047b2-8e2f-4626-ad49-643e5db979d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:00.413311 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c047b2-8e2f-4626-ad49-643e5db979d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:00.413982 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c047b2-8e2f-4626-ad49-643e5db979d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:00.416027 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c047b2-8e2f-4626-ad49-643e5db979d2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:00.592205 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1387/2755] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:23:00 2022] POST /identity/v3/users => generated 291 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:23:00.596562 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e2e45-c527-44ca-8b2f-9f2e22151143 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:00.607313 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e2e45-c527-44ca-8b2f-9f2e22151143 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:00.612379 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f4067-359e-4cbc-80d2-671078d04339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:00.612988 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f4067-359e-4cbc-80d2-671078d04339 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:00.613264 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f4067-359e-4cbc-80d2-671078d04339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:00.613562 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f4067-359e-4cbc-80d2-671078d04339 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:00.614171 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f4067-359e-4cbc-80d2-671078d04339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:00.615800 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f4067-359e-4cbc-80d2-671078d04339 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:00.627586 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1369/2756] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:23:00.632982 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c047b2-8e2f-4626-ad49-643e5db979d2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:00.645010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c047b2-8e2f-4626-ad49-643e5db979d2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:00.650844 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3a51d1-5f85-4077-8a12-7eb281f9c9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:00.651448 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a51d1-5f85-4077-8a12-7eb281f9c9c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:00.651944 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a51d1-5f85-4077-8a12-7eb281f9c9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:00.652080 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a51d1-5f85-4077-8a12-7eb281f9c9c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:00.652714 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3a51d1-5f85-4077-8a12-7eb281f9c9c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:00.654339 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3a51d1-5f85-4077-8a12-7eb281f9c9c5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:00.666637 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1388/2757] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:23:00.670602 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f4067-359e-4cbc-80d2-671078d04339 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:00.682661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f4067-359e-4cbc-80d2-671078d04339 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:00.687392 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ca7d1-e6ef-4ee0-b04e-cfdf857239e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:00.688042 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ca7d1-e6ef-4ee0-b04e-cfdf857239e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:00.688319 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ca7d1-e6ef-4ee0-b04e-cfdf857239e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:00.688588 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ca7d1-e6ef-4ee0-b04e-cfdf857239e3 admin admin] PATH_INFO: `/v3/projects/a49a6fc2c18d44b8a8b82307021355fd/users/0d7d26ea8e26466f91b1905aa1e2e156/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:00.706228 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ca7d1-e6ef-4ee0-b04e-cfdf857239e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a49a6fc2c18d44b8a8b82307021355fd, user_id=0d7d26ea8e26466f91b1905aa1e2e156, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:00.712225 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ca7d1-e6ef-4ee0-b04e-cfdf857239e3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:00.808977 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1370/2758] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:23:00 2022] PUT /identity/v3/projects/a49a6fc2c18d44b8a8b82307021355fd/users/0d7d26ea8e26466f91b1905aa1e2e156/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:23:00.814042 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17dd58-eb24-44ac-b29a-ae40e15a44f5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:00.814325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17dd58-eb24-44ac-b29a-ae40e15a44f5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:00.814632 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17dd58-eb24-44ac-b29a-ae40e15a44f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:00.834082 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ff17dd58-eb24-44ac-b29a-ae40e15a44f5 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:23:00.835379 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ff17dd58-eb24-44ac-b29a-ae40e15a44f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:00.895672 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1389/2759] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:23:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:23:00.906056 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ca7d1-e6ef-4ee0-b04e-cfdf857239e3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:00.907773 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c83ca7d1-e6ef-4ee0-b04e-cfdf857239e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:00.919986 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:23:00.967180 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ca7d1-e6ef-4ee0-b04e-cfdf857239e3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:00.971122 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a944c6-aeb5-47a3-880f-1844c9109490 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:00.971738 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a944c6-aeb5-47a3-880f-1844c9109490 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:00.972054 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a944c6-aeb5-47a3-880f-1844c9109490 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:00.972324 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a944c6-aeb5-47a3-880f-1844c9109490 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:00.981353 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a944c6-aeb5-47a3-880f-1844c9109490 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:00.982803 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a944c6-aeb5-47a3-880f-1844c9109490 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:00.992060 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1371/2760] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:23:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:23:02.399940 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe538a6-b5bf-475b-9f29-dcccef3807aa None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:02.401557 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe538a6-b5bf-475b-9f29-dcccef3807aa None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:02.402010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe538a6-b5bf-475b-9f29-dcccef3807aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:02.431479 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-afe538a6-b5bf-475b-9f29-dcccef3807aa None None] MFA Rules not processed for user `0d7d26ea8e26466f91b1905aa1e2e156`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:23:02.432785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-afe538a6-b5bf-475b-9f29-dcccef3807aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:02.438500 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:23:02.509353 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1390/2761] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:23:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:23:02.520645 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a944c6-aeb5-47a3-880f-1844c9109490 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:02.522017 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-71a944c6-aeb5-47a3-880f-1844c9109490 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:02.587712 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a944c6-aeb5-47a3-880f-1844c9109490 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:02.592084 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0cebaa-9b9f-406e-bf68-bb93494d12fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:02.592697 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cebaa-9b9f-406e-bf68-bb93494d12fe service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:02.592970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cebaa-9b9f-406e-bf68-bb93494d12fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:02.593239 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cebaa-9b9f-406e-bf68-bb93494d12fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:02.603168 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0cebaa-9b9f-406e-bf68-bb93494d12fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:02.604957 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0cebaa-9b9f-406e-bf68-bb93494d12fe service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:02.615056 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1372/2762] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:23:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:23:05.447055 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa31cb-9f33-4c45-9219-00019282eb86 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:05.447610 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa31cb-9f33-4c45-9219-00019282eb86 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:05.447713 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa31cb-9f33-4c45-9219-00019282eb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:05.467263 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-94aa31cb-9f33-4c45-9219-00019282eb86 None None] MFA Rules not processed for user `0d7d26ea8e26466f91b1905aa1e2e156`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:23:05.468463 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-94aa31cb-9f33-4c45-9219-00019282eb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:05.475586 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1391/2763] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:23:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:23:08.223243 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0cebaa-9b9f-406e-bf68-bb93494d12fe service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:08.223243 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-be0cebaa-9b9f-406e-bf68-bb93494d12fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:08.308946 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0cebaa-9b9f-406e-bf68-bb93494d12fe service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:08.314872 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f4bca-65c8-47c3-ab94-82ef2c898927 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cd39d9a583a4d73a5654017be106019', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:08.318009 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f4bca-65c8-47c3-ab94-82ef2c898927 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:08.318009 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f4bca-65c8-47c3-ab94-82ef2c898927 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:08.318009 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f4bca-65c8-47c3-ab94-82ef2c898927 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:08.318009 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e55f4bca-65c8-47c3-ab94-82ef2c898927 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:08.330869 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f4bca-65c8-47c3-ab94-82ef2c898927 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:08.332488 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f4bca-65c8-47c3-ab94-82ef2c898927 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:08.343962 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1373/2764] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:23:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:23:12.679322 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa31cb-9f33-4c45-9219-00019282eb86 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:12.681173 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-94aa31cb-9f33-4c45-9219-00019282eb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:12.703627 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa31cb-9f33-4c45-9219-00019282eb86 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:12.707923 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db58ab7c-1465-4df8-b953-9716e61a7d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:12.708545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58ab7c-1465-4df8-b953-9716e61a7d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:12.708819 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58ab7c-1465-4df8-b953-9716e61a7d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:12.709085 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58ab7c-1465-4df8-b953-9716e61a7d32 admin admin] PATH_INFO: `/v3/users/cd18b8117c724c72a36348bcc5aaae7f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:12.710249 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db58ab7c-1465-4df8-b953-9716e61a7d32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd18b8117c724c72a36348bcc5aaae7f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:12.712398 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db58ab7c-1465-4df8-b953-9716e61a7d32 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:12.901829 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-db58ab7c-1465-4df8-b953-9716e61a7d32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd18b8117c724c72a36348bcc5aaae7f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:13.059762 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-db58ab7c-1465-4df8-b953-9716e61a7d32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd18b8117c724c72a36348bcc5aaae7f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:13.068531 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-db58ab7c-1465-4df8-b953-9716e61a7d32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd18b8117c724c72a36348bcc5aaae7f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:13.082589 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-db58ab7c-1465-4df8-b953-9716e61a7d32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd18b8117c724c72a36348bcc5aaae7f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:13.083334 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-db58ab7c-1465-4df8-b953-9716e61a7d32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd18b8117c724c72a36348bcc5aaae7f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:13.089385 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1392/2765] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:23:12 2022] DELETE /identity/v3/users/cd18b8117c724c72a36348bcc5aaae7f => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:23:13.382174 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f4bca-65c8-47c3-ab94-82ef2c898927 service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:13.393585 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f4bca-65c8-47c3-ab94-82ef2c898927 service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:13.426504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:23:13.463975 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e3400-bf15-44ea-afa9-932070135bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:13.464601 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e3400-bf15-44ea-afa9-932070135bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:13.464873 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e3400-bf15-44ea-afa9-932070135bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:13.465144 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e3400-bf15-44ea-afa9-932070135bb6 admin admin] PATH_INFO: `/v3/projects/7feac9a147e3467a87882322f37821f4` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:13.466287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e3400-bf15-44ea-afa9-932070135bb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7feac9a147e3467a87882322f37821f4)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:13.468207 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e3400-bf15-44ea-afa9-932070135bb6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:13.604307 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-886e3400-bf15-44ea-afa9-932070135bb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7feac9a147e3467a87882322f37821f4', 'request_id': 'req-886e3400-bf15-44ea-afa9-932070135bb6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:13.688531 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-886e3400-bf15-44ea-afa9-932070135bb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7feac9a147e3467a87882322f37821f4', 'request_id': 'req-886e3400-bf15-44ea-afa9-932070135bb6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:13.695970 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-886e3400-bf15-44ea-afa9-932070135bb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7feac9a147e3467a87882322f37821f4', 'request_id': 'req-886e3400-bf15-44ea-afa9-932070135bb6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:13.712846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-886e3400-bf15-44ea-afa9-932070135bb6 admin admin] The token cache is being invalidate because project 7feac9a147e3467a87882322f37821f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:23:13.712846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-886e3400-bf15-44ea-afa9-932070135bb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:13.712846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-886e3400-bf15-44ea-afa9-932070135bb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:13.715586 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1374/2766] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:23:13 2022] DELETE /identity/v3/projects/7feac9a147e3467a87882322f37821f4 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:23:13.719868 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db58ab7c-1465-4df8-b953-9716e61a7d32 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:13.721294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-db58ab7c-1465-4df8-b953-9716e61a7d32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:13.737405 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db58ab7c-1465-4df8-b953-9716e61a7d32 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:13.796494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e469ab-9d7c-421c-9701-7b76973d6ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:13.797035 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e469ab-9d7c-421c-9701-7b76973d6ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:13.797313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e469ab-9d7c-421c-9701-7b76973d6ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:13.797642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e469ab-9d7c-421c-9701-7b76973d6ec9 admin admin] PATH_INFO: `/v3/users/59800960b5a44ca78d5cb67fee10d366` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:13.798958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e469ab-9d7c-421c-9701-7b76973d6ec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59800960b5a44ca78d5cb67fee10d366)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:13.800885 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e469ab-9d7c-421c-9701-7b76973d6ec9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:13.815490 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:23:14.033595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-96e469ab-9d7c-421c-9701-7b76973d6ec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59800960b5a44ca78d5cb67fee10d366'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:14.140027 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-96e469ab-9d7c-421c-9701-7b76973d6ec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59800960b5a44ca78d5cb67fee10d366'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:14.149634 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-96e469ab-9d7c-421c-9701-7b76973d6ec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59800960b5a44ca78d5cb67fee10d366'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:14.161496 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-96e469ab-9d7c-421c-9701-7b76973d6ec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59800960b5a44ca78d5cb67fee10d366'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:14.162227 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-96e469ab-9d7c-421c-9701-7b76973d6ec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59800960b5a44ca78d5cb67fee10d366'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:14.168531 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1393/2767] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:23:13 2022] DELETE /identity/v3/users/59800960b5a44ca78d5cb67fee10d366 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:23:14.542653 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e3400-bf15-44ea-afa9-932070135bb6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:14.544162 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-886e3400-bf15-44ea-afa9-932070135bb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:14.567589 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e3400-bf15-44ea-afa9-932070135bb6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:14.662068 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52513e7-832f-4975-9643-a2ca0842eb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:14.662918 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52513e7-832f-4975-9643-a2ca0842eb42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:14.663353 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52513e7-832f-4975-9643-a2ca0842eb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:14.663682 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52513e7-832f-4975-9643-a2ca0842eb42 admin admin] PATH_INFO: `/v3/projects/0e92e33a696c4473b35334d82030528f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:14.666778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52513e7-832f-4975-9643-a2ca0842eb42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e92e33a696c4473b35334d82030528f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:14.666996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52513e7-832f-4975-9643-a2ca0842eb42 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:14.794523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a52513e7-832f-4975-9643-a2ca0842eb42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e92e33a696c4473b35334d82030528f', 'request_id': 'req-a52513e7-832f-4975-9643-a2ca0842eb42'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:14.865315 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a52513e7-832f-4975-9643-a2ca0842eb42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e92e33a696c4473b35334d82030528f', 'request_id': 'req-a52513e7-832f-4975-9643-a2ca0842eb42'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:14.866940 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:23:14.875316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a52513e7-832f-4975-9643-a2ca0842eb42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e92e33a696c4473b35334d82030528f', 'request_id': 'req-a52513e7-832f-4975-9643-a2ca0842eb42'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:14.886775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a52513e7-832f-4975-9643-a2ca0842eb42 admin admin] The token cache is being invalidate because project 0e92e33a696c4473b35334d82030528f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:23:14.887053 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a52513e7-832f-4975-9643-a2ca0842eb42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:14.887708 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a52513e7-832f-4975-9643-a2ca0842eb42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:14.889185 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1375/2768] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:23:14 2022] DELETE /identity/v3/projects/0e92e33a696c4473b35334d82030528f => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:23:14.894839 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4f817-881a-419e-91dc-10dbe9af775d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:14.895335 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4f817-881a-419e-91dc-10dbe9af775d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:14.895445 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4f817-881a-419e-91dc-10dbe9af775d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:14.915939 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7bd4f817-881a-419e-91dc-10dbe9af775d None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:23:14.917161 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7bd4f817-881a-419e-91dc-10dbe9af775d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:14.980016 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1394/2769] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:23:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:23:14.989556 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35624fe2-b186-441b-b4b0-476ffc64f7e5 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:14.989907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35624fe2-b186-441b-b4b0-476ffc64f7e5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:14.992714 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35624fe2-b186-441b-b4b0-476ffc64f7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:15.010331 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-35624fe2-b186-441b-b4b0-476ffc64f7e5 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:23:15.011569 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-35624fe2-b186-441b-b4b0-476ffc64f7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:15.021184 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1376/2770] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:23:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:23:15.025919 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4f817-881a-419e-91dc-10dbe9af775d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:15.036928 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4f817-881a-419e-91dc-10dbe9af775d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:15.041617 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0c013-3e89-4de4-b790-f5410f390eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:15.042214 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0c013-3e89-4de4-b790-f5410f390eba admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:15.042527 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0c013-3e89-4de4-b790-f5410f390eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:15.042799 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0c013-3e89-4de4-b790-f5410f390eba admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:15.043441 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0c013-3e89-4de4-b790-f5410f390eba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:23:15.043745 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0c013-3e89-4de4-b790-f5410f390eba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:15.045331 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0c013-3e89-4de4-b790-f5410f390eba admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:15.057266 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1395/2771] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:23:15.061777 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35624fe2-b186-441b-b4b0-476ffc64f7e5 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:15.071243 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35624fe2-b186-441b-b4b0-476ffc64f7e5 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:15.076281 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc2fd2-0306-435d-a2c9-db198c4c1f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:15.076903 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc2fd2-0306-435d-a2c9-db198c4c1f86 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:15.077179 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc2fd2-0306-435d-a2c9-db198c4c1f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:15.077449 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc2fd2-0306-435d-a2c9-db198c4c1f86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:15.078098 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc2fd2-0306-435d-a2c9-db198c4c1f86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:15.080700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc2fd2-0306-435d-a2c9-db198c4c1f86 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:15.229004 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1377/2772] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:23:15 2022] POST /identity/v3/projects => generated 375 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:23:15.233743 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0c013-3e89-4de4-b790-f5410f390eba admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:15.245375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0c013-3e89-4de4-b790-f5410f390eba admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:15.305021 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0975275-2631-419f-ba37-5d7852b06dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:15.305645 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0975275-2631-419f-ba37-5d7852b06dbc admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:15.305931 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0975275-2631-419f-ba37-5d7852b06dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:15.306283 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0975275-2631-419f-ba37-5d7852b06dbc admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:15.307018 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0975275-2631-419f-ba37-5d7852b06dbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:15.308954 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0975275-2631-419f-ba37-5d7852b06dbc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:15.420312 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1396/2773] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:23:15 2022] POST /identity/v3/users => generated 288 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:23:15.425128 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc2fd2-0306-435d-a2c9-db198c4c1f86 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:15.436131 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc2fd2-0306-435d-a2c9-db198c4c1f86 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:15.442382 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69fc206-7559-493d-9977-1d6cd2627f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:15.443230 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fc206-7559-493d-9977-1d6cd2627f04 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:15.443565 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fc206-7559-493d-9977-1d6cd2627f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:15.443834 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fc206-7559-493d-9977-1d6cd2627f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:15.444553 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69fc206-7559-493d-9977-1d6cd2627f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:15.446286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69fc206-7559-493d-9977-1d6cd2627f04 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:15.458162 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1378/2774] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:23:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:23:15.462315 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0975275-2631-419f-ba37-5d7852b06dbc admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:15.472502 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0975275-2631-419f-ba37-5d7852b06dbc admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:15.477234 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f5583e-4dab-4527-9e67-ae1d7ee210e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:15.477834 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5583e-4dab-4527-9e67-ae1d7ee210e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:15.478111 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5583e-4dab-4527-9e67-ae1d7ee210e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:15.478405 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5583e-4dab-4527-9e67-ae1d7ee210e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:15.479103 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f5583e-4dab-4527-9e67-ae1d7ee210e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:15.480638 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f5583e-4dab-4527-9e67-ae1d7ee210e6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:15.483127 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:23:15.495461 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1397/2775] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:23:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:23:15.504826 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69fc206-7559-493d-9977-1d6cd2627f04 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:15.515824 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69fc206-7559-493d-9977-1d6cd2627f04 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:15.521336 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf674c-dd5f-44b9-b52f-6331477e0981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:15.521996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf674c-dd5f-44b9-b52f-6331477e0981 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:15.522274 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf674c-dd5f-44b9-b52f-6331477e0981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:15.522573 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf674c-dd5f-44b9-b52f-6331477e0981 admin admin] PATH_INFO: `/v3/projects/0b3c98f4ccc14a0d9bff87f9647671d0/users/4deb36695796426db7761558dd90677e/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:15.537623 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf674c-dd5f-44b9-b52f-6331477e0981 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b3c98f4ccc14a0d9bff87f9647671d0, user_id=4deb36695796426db7761558dd90677e, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:15.540303 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf674c-dd5f-44b9-b52f-6331477e0981 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:15.638289 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1379/2776] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:23:15 2022] PUT /identity/v3/projects/0b3c98f4ccc14a0d9bff87f9647671d0/users/4deb36695796426db7761558dd90677e/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:23:15.648117 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f6edb-88e0-40b6-ad02-1cfeeb9a441e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:15.648671 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f6edb-88e0-40b6-ad02-1cfeeb9a441e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:15.649134 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f6edb-88e0-40b6-ad02-1cfeeb9a441e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:15.679858 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0e9f6edb-88e0-40b6-ad02-1cfeeb9a441e None None] MFA Rules not processed for user `4deb36695796426db7761558dd90677e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:23:15.681101 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0e9f6edb-88e0-40b6-ad02-1cfeeb9a441e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:15.753346 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1398/2777] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:23:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:23:15.758121 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf674c-dd5f-44b9-b52f-6331477e0981 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:15.822769 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf674c-dd5f-44b9-b52f-6331477e0981 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:15.827511 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6c917-a14b-486c-a90f-328f8f5dd1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:15.828112 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6c917-a14b-486c-a90f-328f8f5dd1b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:15.828388 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6c917-a14b-486c-a90f-328f8f5dd1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:15.828662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6c917-a14b-486c-a90f-328f8f5dd1b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:15.829277 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6c917-a14b-486c-a90f-328f8f5dd1b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:15.831467 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6c917-a14b-486c-a90f-328f8f5dd1b3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:15.919179 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1380/2778] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:23:15 2022] POST /identity/v3/projects => generated 377 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:23:15.923278 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f6edb-88e0-40b6-ad02-1cfeeb9a441e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:15.934531 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f6edb-88e0-40b6-ad02-1cfeeb9a441e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:15.992181 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183dbd1d-9e5e-4693-a34a-50754a450352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:15.992807 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dbd1d-9e5e-4693-a34a-50754a450352 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:15.993081 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dbd1d-9e5e-4693-a34a-50754a450352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:15.993350 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dbd1d-9e5e-4693-a34a-50754a450352 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:15.993956 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183dbd1d-9e5e-4693-a34a-50754a450352 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:15.995904 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183dbd1d-9e5e-4693-a34a-50754a450352 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:16.123674 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1399/2779] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:23:15 2022] POST /identity/v3/users => generated 289 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:23:16.128456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6c917-a14b-486c-a90f-328f8f5dd1b3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:16.138160 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6c917-a14b-486c-a90f-328f8f5dd1b3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:16.142996 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccc8f8-2af0-45f6-b336-a40f43b26446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:16.143709 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccc8f8-2af0-45f6-b336-a40f43b26446 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:16.143994 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccc8f8-2af0-45f6-b336-a40f43b26446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:16.144247 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccc8f8-2af0-45f6-b336-a40f43b26446 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:16.144880 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ccc8f8-2af0-45f6-b336-a40f43b26446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:16.146507 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ccc8f8-2af0-45f6-b336-a40f43b26446 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:16.156826 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1381/2780] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:23:16.161385 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183dbd1d-9e5e-4693-a34a-50754a450352 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:16.171712 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183dbd1d-9e5e-4693-a34a-50754a450352 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:16.184013 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e4acf1-a4e8-4c46-9a01-9136ef3188c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:16.184618 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4acf1-a4e8-4c46-9a01-9136ef3188c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:16.184891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4acf1-a4e8-4c46-9a01-9136ef3188c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:16.185161 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4acf1-a4e8-4c46-9a01-9136ef3188c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:16.185803 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e4acf1-a4e8-4c46-9a01-9136ef3188c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:16.187447 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e4acf1-a4e8-4c46-9a01-9136ef3188c9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:16.197794 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1400/2781] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:23:16.202193 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccc8f8-2af0-45f6-b336-a40f43b26446 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:16.212366 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccc8f8-2af0-45f6-b336-a40f43b26446 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:16.217036 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b85436-ba65-4734-84f2-d6570074fc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:16.217662 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b85436-ba65-4734-84f2-d6570074fc03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:16.217931 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b85436-ba65-4734-84f2-d6570074fc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:16.218189 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b85436-ba65-4734-84f2-d6570074fc03 admin admin] PATH_INFO: `/v3/projects/b4ffa7d9d48249f79dddab021ecfa4de/users/2675ab56863e42aba22171aa5cb49416/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:16.233505 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b85436-ba65-4734-84f2-d6570074fc03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4ffa7d9d48249f79dddab021ecfa4de, user_id=2675ab56863e42aba22171aa5cb49416, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:16.236174 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b85436-ba65-4734-84f2-d6570074fc03 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:16.358288 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1382/2782] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:23:16 2022] PUT /identity/v3/projects/b4ffa7d9d48249f79dddab021ecfa4de/users/2675ab56863e42aba22171aa5cb49416/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:23:16.369488 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac08a39-da98-42a9-b8f3-823e34a0458d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:16.369777 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac08a39-da98-42a9-b8f3-823e34a0458d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:16.370054 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac08a39-da98-42a9-b8f3-823e34a0458d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:16.402056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-dac08a39-da98-42a9-b8f3-823e34a0458d None None] MFA Rules not processed for user `2675ab56863e42aba22171aa5cb49416`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:23:16.403247 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-dac08a39-da98-42a9-b8f3-823e34a0458d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:16.461151 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:23:16.471464 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1401/2783] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:23:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:23:16.476292 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b85436-ba65-4734-84f2-d6570074fc03 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:16.518138 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:23:16.546626 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b85436-ba65-4734-84f2-d6570074fc03 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:16.551945 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628640cb-f684-4169-9886-c0a2fd74d7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:16.552552 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628640cb-f684-4169-9886-c0a2fd74d7b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:16.552821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628640cb-f684-4169-9886-c0a2fd74d7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:16.553085 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628640cb-f684-4169-9886-c0a2fd74d7b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:16.553713 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628640cb-f684-4169-9886-c0a2fd74d7b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:16.555753 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628640cb-f684-4169-9886-c0a2fd74d7b1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:16.692503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac08a39-da98-42a9-b8f3-823e34a0458d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:16.694619 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1383/2784] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:23:16 2022] POST /identity/v3/projects => generated 375 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:23:16.708114 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac08a39-da98-42a9-b8f3-823e34a0458d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:16.807887 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd35f61-a1a5-46b2-8326-bbb38343ab95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:16.807887 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd35f61-a1a5-46b2-8326-bbb38343ab95 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:16.807887 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd35f61-a1a5-46b2-8326-bbb38343ab95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:16.807887 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd35f61-a1a5-46b2-8326-bbb38343ab95 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:16.808229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd35f61-a1a5-46b2-8326-bbb38343ab95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:16.808229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd35f61-a1a5-46b2-8326-bbb38343ab95 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:16.973300 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1402/2785] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:23:16 2022] POST /identity/v3/users => generated 294 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:23:16.974687 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628640cb-f684-4169-9886-c0a2fd74d7b1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:16.985276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628640cb-f684-4169-9886-c0a2fd74d7b1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:16.991630 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e628714e-dc03-403c-8873-747bf3dfaa2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:16.992242 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628714e-dc03-403c-8873-747bf3dfaa2a admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:16.992514 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628714e-dc03-403c-8873-747bf3dfaa2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:16.992775 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628714e-dc03-403c-8873-747bf3dfaa2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:16.993420 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e628714e-dc03-403c-8873-747bf3dfaa2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:16.995049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e628714e-dc03-403c-8873-747bf3dfaa2a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:17.006846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1384/2786] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:23:17.011317 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd35f61-a1a5-46b2-8326-bbb38343ab95 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:17.022144 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd35f61-a1a5-46b2-8326-bbb38343ab95 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:17.026776 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde7971-4fde-4712-8cb4-6fd72b21d22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:17.031127 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde7971-4fde-4712-8cb4-6fd72b21d22a admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:17.031127 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde7971-4fde-4712-8cb4-6fd72b21d22a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:17.031127 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde7971-4fde-4712-8cb4-6fd72b21d22a admin admin] PATH_INFO: `/v3/projects/650014b43c5d42219ef02133e6ef60a1/users/44a2cc9f8fa441f68c7ba8c7c2968007/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:17.042817 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dde7971-4fde-4712-8cb4-6fd72b21d22a admin admin] RBAC: Authorizing `identity:create_grant(project_id=650014b43c5d42219ef02133e6ef60a1, user_id=44a2cc9f8fa441f68c7ba8c7c2968007, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:17.045510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dde7971-4fde-4712-8cb4-6fd72b21d22a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:17.261043 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1403/2787] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:23:17 2022] PUT /identity/v3/projects/650014b43c5d42219ef02133e6ef60a1/users/44a2cc9f8fa441f68c7ba8c7c2968007/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:23:17.265105 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e628714e-dc03-403c-8873-747bf3dfaa2a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:17.275262 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e628714e-dc03-403c-8873-747bf3dfaa2a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:17.334998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f711e7-3292-404a-8fed-459420e63b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:17.335605 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f711e7-3292-404a-8fed-459420e63b9d admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:17.335901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f711e7-3292-404a-8fed-459420e63b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:17.336160 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f711e7-3292-404a-8fed-459420e63b9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:17.336803 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f711e7-3292-404a-8fed-459420e63b9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:17.338413 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f711e7-3292-404a-8fed-459420e63b9d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:17.349158 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1385/2788] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:23:17.353353 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde7971-4fde-4712-8cb4-6fd72b21d22a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:17.367967 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde7971-4fde-4712-8cb4-6fd72b21d22a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:17.375292 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5857e-383f-4fc9-965f-71a2e388f3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:17.375976 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5857e-383f-4fc9-965f-71a2e388f3d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:17.376253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5857e-383f-4fc9-965f-71a2e388f3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:17.376541 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5857e-383f-4fc9-965f-71a2e388f3d5 admin admin] PATH_INFO: `/v3/projects/650014b43c5d42219ef02133e6ef60a1/users/44a2cc9f8fa441f68c7ba8c7c2968007/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:17.379314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5857e-383f-4fc9-965f-71a2e388f3d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=650014b43c5d42219ef02133e6ef60a1, user_id=44a2cc9f8fa441f68c7ba8c7c2968007, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:17.382255 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5857e-383f-4fc9-965f-71a2e388f3d5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:17.394378 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1404/2789] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:23:17 2022] PUT /identity/v3/projects/650014b43c5d42219ef02133e6ef60a1/users/44a2cc9f8fa441f68c7ba8c7c2968007/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:23:17.411283 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd290d3-7d67-4284-bc8e-32e41581c1f4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:17.411621 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd290d3-7d67-4284-bc8e-32e41581c1f4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:17.411896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd290d3-7d67-4284-bc8e-32e41581c1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:17.443484 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5cd290d3-7d67-4284-bc8e-32e41581c1f4 None None] MFA Rules not processed for user `44a2cc9f8fa441f68c7ba8c7c2968007`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:23:17.444883 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5cd290d3-7d67-4284-bc8e-32e41581c1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:17.517826 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1386/2790] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:23:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:23:17.529148 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5857e-383f-4fc9-965f-71a2e388f3d5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:17.530893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-12a5857e-383f-4fc9-965f-71a2e388f3d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:17.590311 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5857e-383f-4fc9-965f-71a2e388f3d5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:17.593930 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5b2f3-fade-4908-99d7-e2eae0270ff4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:17.594559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5b2f3-fade-4908-99d7-e2eae0270ff4 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:17.594855 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5b2f3-fade-4908-99d7-e2eae0270ff4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:17.595120 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5b2f3-fade-4908-99d7-e2eae0270ff4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:17.652737 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e5b2f3-fade-4908-99d7-e2eae0270ff4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:17.654199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e5b2f3-fade-4908-99d7-e2eae0270ff4 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:17.665581 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1405/2791] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:23:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:23:19.978818 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd290d3-7d67-4284-bc8e-32e41581c1f4 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:19.980285 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5cd290d3-7d67-4284-bc8e-32e41581c1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:19.992991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd290d3-7d67-4284-bc8e-32e41581c1f4 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:19.996608 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614b974-8859-45c6-909b-35a6f8e73b23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:19.997202 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614b974-8859-45c6-909b-35a6f8e73b23 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:19.997478 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614b974-8859-45c6-909b-35a6f8e73b23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:19.997744 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614b974-8859-45c6-909b-35a6f8e73b23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:20.023489 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:23:20.057858 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6614b974-8859-45c6-909b-35a6f8e73b23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:20.059332 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6614b974-8859-45c6-909b-35a6f8e73b23 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:20.069516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1387/2792] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:23:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:23:21.829345 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5b2f3-fade-4908-99d7-e2eae0270ff4 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:21.838256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:23:21.847163 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5b2f3-fade-4908-99d7-e2eae0270ff4 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:21.851716 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2fc6f-a3f7-4add-9d8b-609eca6304d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:21.852334 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2fc6f-a3f7-4add-9d8b-609eca6304d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:21.852610 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2fc6f-a3f7-4add-9d8b-609eca6304d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:21.854049 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2fc6f-a3f7-4add-9d8b-609eca6304d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:21.868257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2fc6f-a3f7-4add-9d8b-609eca6304d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:21.870748 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2fc6f-a3f7-4add-9d8b-609eca6304d6 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:21.894575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1406/2793] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:23:21 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:23:26.638330 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614b974-8859-45c6-909b-35a6f8e73b23 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:26.706067 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614b974-8859-45c6-909b-35a6f8e73b23 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:26.710781 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89431364-1ebd-4033-bc8d-64394fd3b534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:26.711434 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89431364-1ebd-4033-bc8d-64394fd3b534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:26.711712 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89431364-1ebd-4033-bc8d-64394fd3b534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:26.711987 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89431364-1ebd-4033-bc8d-64394fd3b534 admin admin] PATH_INFO: `/v3/users/4deb36695796426db7761558dd90677e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:26.713191 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89431364-1ebd-4033-bc8d-64394fd3b534 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4deb36695796426db7761558dd90677e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:26.715212 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89431364-1ebd-4033-bc8d-64394fd3b534 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:26.921016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-89431364-1ebd-4033-bc8d-64394fd3b534 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4deb36695796426db7761558dd90677e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:26.997720 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-89431364-1ebd-4033-bc8d-64394fd3b534 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4deb36695796426db7761558dd90677e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:27.018977 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-89431364-1ebd-4033-bc8d-64394fd3b534 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4deb36695796426db7761558dd90677e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:27.019763 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-89431364-1ebd-4033-bc8d-64394fd3b534 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4deb36695796426db7761558dd90677e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:27.020378 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-89431364-1ebd-4033-bc8d-64394fd3b534 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4deb36695796426db7761558dd90677e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:27.035788 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1388/2794] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:23:26 2022] DELETE /identity/v3/users/4deb36695796426db7761558dd90677e => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:23:27.040644 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696ac4a-f572-4356-a511-93a7a1e8f8bc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:27.041444 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696ac4a-f572-4356-a511-93a7a1e8f8bc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:27.041659 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696ac4a-f572-4356-a511-93a7a1e8f8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:27.063183 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e696ac4a-f572-4356-a511-93a7a1e8f8bc None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:23:27.065714 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e696ac4a-f572-4356-a511-93a7a1e8f8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:27.131177 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1407/2795] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:23:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:23:27.141446 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89431364-1ebd-4033-bc8d-64394fd3b534 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:27.142962 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-89431364-1ebd-4033-bc8d-64394fd3b534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:27.198720 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:23:27.212298 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89431364-1ebd-4033-bc8d-64394fd3b534 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:27.216484 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96cee9-db32-4bb9-a051-a970c02b26a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:27.217135 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96cee9-db32-4bb9-a051-a970c02b26a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:27.217409 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96cee9-db32-4bb9-a051-a970c02b26a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:27.217676 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96cee9-db32-4bb9-a051-a970c02b26a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:27.229728 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e96cee9-db32-4bb9-a051-a970c02b26a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:27.231385 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e96cee9-db32-4bb9-a051-a970c02b26a7 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:27.244507 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1389/2796] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:23:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:23:27.764242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696ac4a-f572-4356-a511-93a7a1e8f8bc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:27.766027 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e696ac4a-f572-4356-a511-93a7a1e8f8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:27.793404 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696ac4a-f572-4356-a511-93a7a1e8f8bc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:27.800010 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf29f3-8576-45d5-bf84-f791a8fbc438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:27.801218 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf29f3-8576-45d5-bf84-f791a8fbc438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:27.801635 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf29f3-8576-45d5-bf84-f791a8fbc438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:27.802126 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf29f3-8576-45d5-bf84-f791a8fbc438 admin admin] PATH_INFO: `/v3/projects/0b3c98f4ccc14a0d9bff87f9647671d0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:27.804183 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf29f3-8576-45d5-bf84-f791a8fbc438 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b3c98f4ccc14a0d9bff87f9647671d0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:27.808127 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf29f3-8576-45d5-bf84-f791a8fbc438 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:27.957794 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2cbf29f3-8576-45d5-bf84-f791a8fbc438 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b3c98f4ccc14a0d9bff87f9647671d0', 'request_id': 'req-2cbf29f3-8576-45d5-bf84-f791a8fbc438'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:27.963996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2cbf29f3-8576-45d5-bf84-f791a8fbc438 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b3c98f4ccc14a0d9bff87f9647671d0', 'request_id': 'req-2cbf29f3-8576-45d5-bf84-f791a8fbc438'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:28.099567 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2cbf29f3-8576-45d5-bf84-f791a8fbc438 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b3c98f4ccc14a0d9bff87f9647671d0', 'request_id': 'req-2cbf29f3-8576-45d5-bf84-f791a8fbc438'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:28.111408 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2cbf29f3-8576-45d5-bf84-f791a8fbc438 admin admin] The token cache is being invalidate because project 0b3c98f4ccc14a0d9bff87f9647671d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:23:28.111721 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2cbf29f3-8576-45d5-bf84-f791a8fbc438 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:28.112516 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2cbf29f3-8576-45d5-bf84-f791a8fbc438 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:28.113932 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1408/2797] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:23:27 2022] DELETE /identity/v3/projects/0b3c98f4ccc14a0d9bff87f9647671d0 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:23:28.118129 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96cee9-db32-4bb9-a051-a970c02b26a7 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:28.128685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96cee9-db32-4bb9-a051-a970c02b26a7 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:28.185240 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf0e7d-f492-45d8-a4a4-9f690e1ae25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:28.185902 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf0e7d-f492-45d8-a4a4-9f690e1ae25d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:28.186177 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf0e7d-f492-45d8-a4a4-9f690e1ae25d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:28.186494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf0e7d-f492-45d8-a4a4-9f690e1ae25d admin admin] PATH_INFO: `/v3/users/2675ab56863e42aba22171aa5cb49416` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:28.187758 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcf0e7d-f492-45d8-a4a4-9f690e1ae25d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2675ab56863e42aba22171aa5cb49416)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:28.190468 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcf0e7d-f492-45d8-a4a4-9f690e1ae25d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:28.347494 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-0fcf0e7d-f492-45d8-a4a4-9f690e1ae25d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2675ab56863e42aba22171aa5cb49416'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:28.425952 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-0fcf0e7d-f492-45d8-a4a4-9f690e1ae25d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2675ab56863e42aba22171aa5cb49416'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:28.439881 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-0fcf0e7d-f492-45d8-a4a4-9f690e1ae25d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2675ab56863e42aba22171aa5cb49416'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:28.440628 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-0fcf0e7d-f492-45d8-a4a4-9f690e1ae25d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2675ab56863e42aba22171aa5cb49416'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:28.441313 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-0fcf0e7d-f492-45d8-a4a4-9f690e1ae25d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2675ab56863e42aba22171aa5cb49416'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:28.457613 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1390/2798] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:23:28 2022] DELETE /identity/v3/users/2675ab56863e42aba22171aa5cb49416 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:23:29.027966 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf29f3-8576-45d5-bf84-f791a8fbc438 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:29.029385 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2cbf29f3-8576-45d5-bf84-f791a8fbc438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:29.048581 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf29f3-8576-45d5-bf84-f791a8fbc438 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:29.111741 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c15897-9e6a-4602-a0b9-832c7bafe5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:29.112577 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c15897-9e6a-4602-a0b9-832c7bafe5e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:29.112952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c15897-9e6a-4602-a0b9-832c7bafe5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:29.113224 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c15897-9e6a-4602-a0b9-832c7bafe5e0 admin admin] PATH_INFO: `/v3/projects/b4ffa7d9d48249f79dddab021ecfa4de` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:29.114534 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c15897-9e6a-4602-a0b9-832c7bafe5e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4ffa7d9d48249f79dddab021ecfa4de)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:29.116549 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c15897-9e6a-4602-a0b9-832c7bafe5e0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:29.134311 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:23:29.277317 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c5c15897-9e6a-4602-a0b9-832c7bafe5e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4ffa7d9d48249f79dddab021ecfa4de', 'request_id': 'req-c5c15897-9e6a-4602-a0b9-832c7bafe5e0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:29.284401 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c5c15897-9e6a-4602-a0b9-832c7bafe5e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4ffa7d9d48249f79dddab021ecfa4de', 'request_id': 'req-c5c15897-9e6a-4602-a0b9-832c7bafe5e0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:29.368644 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c5c15897-9e6a-4602-a0b9-832c7bafe5e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4ffa7d9d48249f79dddab021ecfa4de', 'request_id': 'req-c5c15897-9e6a-4602-a0b9-832c7bafe5e0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:29.383184 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c5c15897-9e6a-4602-a0b9-832c7bafe5e0 admin admin] The token cache is being invalidate because project b4ffa7d9d48249f79dddab021ecfa4de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:23:29.383468 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c5c15897-9e6a-4602-a0b9-832c7bafe5e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:29.384128 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c5c15897-9e6a-4602-a0b9-832c7bafe5e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:29.385692 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1409/2799] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:23:29 2022] DELETE /identity/v3/projects/b4ffa7d9d48249f79dddab021ecfa4de => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:23:29.389764 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf0e7d-f492-45d8-a4a4-9f690e1ae25d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:29.391245 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0fcf0e7d-f492-45d8-a4a4-9f690e1ae25d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:29.404332 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf0e7d-f492-45d8-a4a4-9f690e1ae25d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:29.460658 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ec844-56ab-47ab-9293-0abe9e49df88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:29.461273 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ec844-56ab-47ab-9293-0abe9e49df88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:29.461547 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ec844-56ab-47ab-9293-0abe9e49df88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:29.461808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ec844-56ab-47ab-9293-0abe9e49df88 admin admin] PATH_INFO: `/v3/users/44a2cc9f8fa441f68c7ba8c7c2968007` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:29.462930 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ec844-56ab-47ab-9293-0abe9e49df88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44a2cc9f8fa441f68c7ba8c7c2968007)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:29.464907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ec844-56ab-47ab-9293-0abe9e49df88 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:29.478154 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:23:29.683367 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-277ec844-56ab-47ab-9293-0abe9e49df88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44a2cc9f8fa441f68c7ba8c7c2968007'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:29.785089 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-277ec844-56ab-47ab-9293-0abe9e49df88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44a2cc9f8fa441f68c7ba8c7c2968007'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:29.797742 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-277ec844-56ab-47ab-9293-0abe9e49df88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44a2cc9f8fa441f68c7ba8c7c2968007'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:29.798429 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-277ec844-56ab-47ab-9293-0abe9e49df88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44a2cc9f8fa441f68c7ba8c7c2968007'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:29.799618 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-277ec844-56ab-47ab-9293-0abe9e49df88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44a2cc9f8fa441f68c7ba8c7c2968007'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:29.811655 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1391/2800] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:23:29 2022] DELETE /identity/v3/users/44a2cc9f8fa441f68c7ba8c7c2968007 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:23:30.305345 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c15897-9e6a-4602-a0b9-832c7bafe5e0 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:30.307538 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c5c15897-9e6a-4602-a0b9-832c7bafe5e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:30.324039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c15897-9e6a-4602-a0b9-832c7bafe5e0 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:30.387538 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa071e7-b009-45af-b352-af7ffac3dfec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:30.388292 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa071e7-b009-45af-b352-af7ffac3dfec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:30.388571 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa071e7-b009-45af-b352-af7ffac3dfec admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:30.388901 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa071e7-b009-45af-b352-af7ffac3dfec admin admin] PATH_INFO: `/v3/projects/650014b43c5d42219ef02133e6ef60a1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:30.390246 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa071e7-b009-45af-b352-af7ffac3dfec admin admin] RBAC: Authorizing `identity:delete_project(project_id=650014b43c5d42219ef02133e6ef60a1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:30.393512 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa071e7-b009-45af-b352-af7ffac3dfec admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:30.538964 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1fa071e7-b009-45af-b352-af7ffac3dfec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '650014b43c5d42219ef02133e6ef60a1', 'request_id': 'req-1fa071e7-b009-45af-b352-af7ffac3dfec'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:30.544971 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1fa071e7-b009-45af-b352-af7ffac3dfec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '650014b43c5d42219ef02133e6ef60a1', 'request_id': 'req-1fa071e7-b009-45af-b352-af7ffac3dfec'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:30.620273 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1fa071e7-b009-45af-b352-af7ffac3dfec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '650014b43c5d42219ef02133e6ef60a1', 'request_id': 'req-1fa071e7-b009-45af-b352-af7ffac3dfec'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:30.634979 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1fa071e7-b009-45af-b352-af7ffac3dfec admin admin] The token cache is being invalidate because project 650014b43c5d42219ef02133e6ef60a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:23:30.638074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1fa071e7-b009-45af-b352-af7ffac3dfec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:30.638074 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1fa071e7-b009-45af-b352-af7ffac3dfec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:30.639420 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1410/2801] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:23:30 2022] DELETE /identity/v3/projects/650014b43c5d42219ef02133e6ef60a1 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:23:30.645266 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7104403-b2b5-4381-8590-da80d320b564 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:30.645839 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7104403-b2b5-4381-8590-da80d320b564 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:30.647141 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7104403-b2b5-4381-8590-da80d320b564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:30.671523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a7104403-b2b5-4381-8590-da80d320b564 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:23:30.672864 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a7104403-b2b5-4381-8590-da80d320b564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:30.747703 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1392/2802] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:23:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:23:30.755764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1ae30-516d-48a6-99b4-a8a8aea6835c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:30.756094 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1ae30-516d-48a6-99b4-a8a8aea6835c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:30.756572 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1ae30-516d-48a6-99b4-a8a8aea6835c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:30.780237 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-63c1ae30-516d-48a6-99b4-a8a8aea6835c None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:23:30.781635 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-63c1ae30-516d-48a6-99b4-a8a8aea6835c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:30.791157 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1411/2803] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:23:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:23:30.797388 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7104403-b2b5-4381-8590-da80d320b564 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:30.808077 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7104403-b2b5-4381-8590-da80d320b564 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:30.812729 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664450d-f234-47d2-b338-c2257ee04b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:30.813330 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664450d-f234-47d2-b338-c2257ee04b1e admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:30.813605 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664450d-f234-47d2-b338-c2257ee04b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:30.813873 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664450d-f234-47d2-b338-c2257ee04b1e admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:30.815515 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b664450d-f234-47d2-b338-c2257ee04b1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:23:30.815853 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b664450d-f234-47d2-b338-c2257ee04b1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:30.817450 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b664450d-f234-47d2-b338-c2257ee04b1e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:30.829094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1393/2804] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:23:30.833600 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1ae30-516d-48a6-99b4-a8a8aea6835c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:30.843900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1ae30-516d-48a6-99b4-a8a8aea6835c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:30.848779 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec53c9-b0b2-467a-938e-ad729e93c1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:30.849386 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec53c9-b0b2-467a-938e-ad729e93c1cd admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:30.849663 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec53c9-b0b2-467a-938e-ad729e93c1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:30.849930 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec53c9-b0b2-467a-938e-ad729e93c1cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:30.850614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec53c9-b0b2-467a-938e-ad729e93c1cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:30.852666 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec53c9-b0b2-467a-938e-ad729e93c1cd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:30.978085 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1412/2805] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:23:30 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:23:30.982451 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664450d-f234-47d2-b338-c2257ee04b1e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:30.994757 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664450d-f234-47d2-b338-c2257ee04b1e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:31.055417 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca2304f-b534-443a-a4ac-a9fb53a96db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:31.056014 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2304f-b534-443a-a4ac-a9fb53a96db4 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:31.056293 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2304f-b534-443a-a4ac-a9fb53a96db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:31.056566 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2304f-b534-443a-a4ac-a9fb53a96db4 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:31.057221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca2304f-b534-443a-a4ac-a9fb53a96db4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:31.059276 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca2304f-b534-443a-a4ac-a9fb53a96db4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:31.146930 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1394/2806] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:23:30 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:23:31.151150 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec53c9-b0b2-467a-938e-ad729e93c1cd admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:31.157918 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:23:31.162097 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec53c9-b0b2-467a-938e-ad729e93c1cd admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:31.167067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3e0ff-908e-4afa-a744-424cf92393ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:31.167657 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3e0ff-908e-4afa-a744-424cf92393ef admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:31.167926 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3e0ff-908e-4afa-a744-424cf92393ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:31.168189 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3e0ff-908e-4afa-a744-424cf92393ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:31.168854 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3e0ff-908e-4afa-a744-424cf92393ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:31.170467 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3e0ff-908e-4afa-a744-424cf92393ef admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:31.180847 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1413/2807] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:23:31.184841 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca2304f-b534-443a-a4ac-a9fb53a96db4 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:31.190765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:23:31.196096 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca2304f-b534-443a-a4ac-a9fb53a96db4 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:31.200801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d4cfa-8aa0-4f48-83da-758457b2d7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:31.202599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d4cfa-8aa0-4f48-83da-758457b2d7a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:31.202599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d4cfa-8aa0-4f48-83da-758457b2d7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:31.202599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d4cfa-8aa0-4f48-83da-758457b2d7a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:31.202599 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d4cfa-8aa0-4f48-83da-758457b2d7a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:31.204238 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d4cfa-8aa0-4f48-83da-758457b2d7a0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:31.215856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1395/2808] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:23:31.220143 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3e0ff-908e-4afa-a744-424cf92393ef admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:31.230176 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3e0ff-908e-4afa-a744-424cf92393ef admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:31.234929 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c25b7ab-fea4-4d69-9900-545698499e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:31.235605 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25b7ab-fea4-4d69-9900-545698499e3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:31.235884 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25b7ab-fea4-4d69-9900-545698499e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:31.236232 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25b7ab-fea4-4d69-9900-545698499e3a admin admin] PATH_INFO: `/v3/projects/0ca8a0c2eb86470cb9e609f05e4ceee7/users/dd1ac5036e16409a9a5cac62fed7d7e7/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:31.251178 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c25b7ab-fea4-4d69-9900-545698499e3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ca8a0c2eb86470cb9e609f05e4ceee7, user_id=dd1ac5036e16409a9a5cac62fed7d7e7, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:31.254400 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c25b7ab-fea4-4d69-9900-545698499e3a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:31.340291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1414/2809] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:23:31 2022] PUT /identity/v3/projects/0ca8a0c2eb86470cb9e609f05e4ceee7/users/dd1ac5036e16409a9a5cac62fed7d7e7/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:23:31.345667 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6b511-be3f-429b-b7a2-a1691f7337d3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:31.346129 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6b511-be3f-429b-b7a2-a1691f7337d3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:31.346458 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6b511-be3f-429b-b7a2-a1691f7337d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:31.365709 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-97d6b511-be3f-429b-b7a2-a1691f7337d3 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:23:31.366937 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-97d6b511-be3f-429b-b7a2-a1691f7337d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:31.425256 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1396/2810] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:23:31.434770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c25b7ab-fea4-4d69-9900-545698499e3a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:31.436509 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1c25b7ab-fea4-4d69-9900-545698499e3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:31.494841 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c25b7ab-fea4-4d69-9900-545698499e3a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:31.498494 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e986fb7-243d-4220-a4b2-9385a399961f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:31.499096 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e986fb7-243d-4220-a4b2-9385a399961f service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:31.499364 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e986fb7-243d-4220-a4b2-9385a399961f service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:31.499625 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e986fb7-243d-4220-a4b2-9385a399961f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:31.509595 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e986fb7-243d-4220-a4b2-9385a399961f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:31.511056 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e986fb7-243d-4220-a4b2-9385a399961f service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:31.520417 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1415/2811] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:23:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:23:35.872437 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6b511-be3f-429b-b7a2-a1691f7337d3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:35.875030 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-97d6b511-be3f-429b-b7a2-a1691f7337d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:35.890860 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6b511-be3f-429b-b7a2-a1691f7337d3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:35.897328 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eae24e-e181-41e0-8191-fdd0e3335895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:35.899138 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eae24e-e181-41e0-8191-fdd0e3335895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:35.899745 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eae24e-e181-41e0-8191-fdd0e3335895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:35.900165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eae24e-e181-41e0-8191-fdd0e3335895 admin admin] PATH_INFO: `/v3/users/b8c54e4edff24f75b28febf51f772672` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:35.902854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eae24e-e181-41e0-8191-fdd0e3335895 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8c54e4edff24f75b28febf51f772672)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:35.906808 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eae24e-e181-41e0-8191-fdd0e3335895 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:36.295246 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-53eae24e-e181-41e0-8191-fdd0e3335895 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8c54e4edff24f75b28febf51f772672'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:36.419705 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-53eae24e-e181-41e0-8191-fdd0e3335895 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8c54e4edff24f75b28febf51f772672'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:36.440643 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-53eae24e-e181-41e0-8191-fdd0e3335895 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8c54e4edff24f75b28febf51f772672'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:36.441388 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-53eae24e-e181-41e0-8191-fdd0e3335895 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8c54e4edff24f75b28febf51f772672'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:36.442031 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-53eae24e-e181-41e0-8191-fdd0e3335895 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8c54e4edff24f75b28febf51f772672'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:36.462926 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1397/2812] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:23:35 2022] DELETE /identity/v3/users/b8c54e4edff24f75b28febf51f772672 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:23:36.466995 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a7f5b-31a5-4fc8-9ef8-03d4970ed570 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:36.469261 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a7f5b-31a5-4fc8-9ef8-03d4970ed570 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:36.469796 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a7f5b-31a5-4fc8-9ef8-03d4970ed570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:36.499183 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ec8a7f5b-31a5-4fc8-9ef8-03d4970ed570 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:23:36.501469 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ec8a7f5b-31a5-4fc8-9ef8-03d4970ed570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:36.593314 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1416/2813] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:23:36.618145 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eae24e-e181-41e0-8191-fdd0e3335895 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:36.618145 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-53eae24e-e181-41e0-8191-fdd0e3335895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:36.714881 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eae24e-e181-41e0-8191-fdd0e3335895 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:36.720074 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07aa64-ab19-49f0-ad30-84623372c2c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:36.721981 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07aa64-ab19-49f0-ad30-84623372c2c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:36.722444 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07aa64-ab19-49f0-ad30-84623372c2c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:36.722729 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07aa64-ab19-49f0-ad30-84623372c2c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:36.745116 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07aa64-ab19-49f0-ad30-84623372c2c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:36.747889 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07aa64-ab19-49f0-ad30-84623372c2c8 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:36.759898 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1398/2814] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:23:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:23:37.388077 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a7f5b-31a5-4fc8-9ef8-03d4970ed570 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:37.389811 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ec8a7f5b-31a5-4fc8-9ef8-03d4970ed570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:37.412682 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a7f5b-31a5-4fc8-9ef8-03d4970ed570 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:37.433814 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ab93e-3e1e-45e6-af3b-16cd019884c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:37.439857 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ab93e-3e1e-45e6-af3b-16cd019884c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:37.440147 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ab93e-3e1e-45e6-af3b-16cd019884c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:37.440427 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ab93e-3e1e-45e6-af3b-16cd019884c9 admin admin] PATH_INFO: `/v3/projects/6d77910ea6a6498da86d470e54ce4fb8` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:37.441929 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ab93e-3e1e-45e6-af3b-16cd019884c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d77910ea6a6498da86d470e54ce4fb8)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:37.443921 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ab93e-3e1e-45e6-af3b-16cd019884c9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:37.611405 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075a19d-9626-4fa5-a6d1-74cab29b7c4b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:37.613009 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075a19d-9626-4fa5-a6d1-74cab29b7c4b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:37.613312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075a19d-9626-4fa5-a6d1-74cab29b7c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:37.634751 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:23:37.644643 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c58ab93e-3e1e-45e6-af3b-16cd019884c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d77910ea6a6498da86d470e54ce4fb8', 'request_id': 'req-c58ab93e-3e1e-45e6-af3b-16cd019884c9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:37.653769 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c58ab93e-3e1e-45e6-af3b-16cd019884c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d77910ea6a6498da86d470e54ce4fb8', 'request_id': 'req-c58ab93e-3e1e-45e6-af3b-16cd019884c9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:37.656413 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c075a19d-9626-4fa5-a6d1-74cab29b7c4b None None] MFA Rules not processed for user `dd1ac5036e16409a9a5cac62fed7d7e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:23:37.659453 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c075a19d-9626-4fa5-a6d1-74cab29b7c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:37.730146 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1399/2815] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:23:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:23:37.734651 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075a19d-9626-4fa5-a6d1-74cab29b7c4b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:37.736110 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c075a19d-9626-4fa5-a6d1-74cab29b7c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:37.785109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c58ab93e-3e1e-45e6-af3b-16cd019884c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d77910ea6a6498da86d470e54ce4fb8', 'request_id': 'req-c58ab93e-3e1e-45e6-af3b-16cd019884c9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:37.793080 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:23:37.804399 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c58ab93e-3e1e-45e6-af3b-16cd019884c9 admin admin] The token cache is being invalidate because project 6d77910ea6a6498da86d470e54ce4fb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:23:37.804807 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c58ab93e-3e1e-45e6-af3b-16cd019884c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:37.805557 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c58ab93e-3e1e-45e6-af3b-16cd019884c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:37.807359 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1417/2816] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:23:37 2022] DELETE /identity/v3/projects/6d77910ea6a6498da86d470e54ce4fb8 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:23:37.812462 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1aca3-ec59-47d5-af46-4d49bf4f5c33 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:37.812752 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1aca3-ec59-47d5-af46-4d49bf4f5c33 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:37.813032 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1aca3-ec59-47d5-af46-4d49bf4f5c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:37.828496 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075a19d-9626-4fa5-a6d1-74cab29b7c4b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:37.834286 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317688f-1b6a-479f-9f1a-9cedaf024af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:37.835020 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317688f-1b6a-479f-9f1a-9cedaf024af6 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:37.835369 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317688f-1b6a-479f-9f1a-9cedaf024af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:37.835644 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f9e1aca3-ec59-47d5-af46-4d49bf4f5c33 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:23:37.835778 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317688f-1b6a-479f-9f1a-9cedaf024af6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:37.836484 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3317688f-1b6a-479f-9f1a-9cedaf024af6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:37.837070 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f9e1aca3-ec59-47d5-af46-4d49bf4f5c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:37.838907 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3317688f-1b6a-479f-9f1a-9cedaf024af6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:37.846190 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1418/2817] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:23:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:23:37.854257 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0498092-860a-4e81-8336-655b395af47d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:37.854565 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0498092-860a-4e81-8336-655b395af47d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:37.854843 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0498092-860a-4e81-8336-655b395af47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:37.873807 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a0498092-860a-4e81-8336-655b395af47d None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:23:37.875053 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a0498092-860a-4e81-8336-655b395af47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:37.885440 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1419/2818] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:23:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:23:37.889818 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0498092-860a-4e81-8336-655b395af47d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:37.899406 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0498092-860a-4e81-8336-655b395af47d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:37.904062 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e8c93-317a-4c2b-b59c-baae232a6d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:37.904642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e8c93-317a-4c2b-b59c-baae232a6d66 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:37.904919 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e8c93-317a-4c2b-b59c-baae232a6d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:37.905183 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e8c93-317a-4c2b-b59c-baae232a6d66 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:37.905819 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e8c93-317a-4c2b-b59c-baae232a6d66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:23:37.906115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e8c93-317a-4c2b-b59c-baae232a6d66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:37.908007 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e8c93-317a-4c2b-b59c-baae232a6d66 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:37.919765 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1420/2819] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:23:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 17:23:37.931963 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e8c93-317a-4c2b-b59c-baae232a6d66 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:37.942585 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e8c93-317a-4c2b-b59c-baae232a6d66 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:37.947531 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd8835e-182e-4978-9576-a32ea4cc75c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:37.948968 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8835e-182e-4978-9576-a32ea4cc75c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:37.949263 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8835e-182e-4978-9576-a32ea4cc75c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:37.949539 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8835e-182e-4978-9576-a32ea4cc75c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:37.950228 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd8835e-182e-4978-9576-a32ea4cc75c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:37.952246 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd8835e-182e-4978-9576-a32ea4cc75c1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:37.958432 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1400/2820] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:23:37 2022] POST /identity/v3/projects => generated 381 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:23:37.961394 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317688f-1b6a-479f-9f1a-9cedaf024af6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:37.963131 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3317688f-1b6a-479f-9f1a-9cedaf024af6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:38.029856 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317688f-1b6a-479f-9f1a-9cedaf024af6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:38.034939 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafde145-37a2-4d88-9ac3-b0fa5de7add3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:38.035549 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafde145-37a2-4d88-9ac3-b0fa5de7add3 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:38.035823 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafde145-37a2-4d88-9ac3-b0fa5de7add3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:38.036092 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafde145-37a2-4d88-9ac3-b0fa5de7add3 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:38.036718 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafde145-37a2-4d88-9ac3-b0fa5de7add3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:38.038739 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafde145-37a2-4d88-9ac3-b0fa5de7add3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:38.049674 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1421/2821] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:23:37 2022] POST /identity/v3/projects => generated 395 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:23:38.054069 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd8835e-182e-4978-9576-a32ea4cc75c1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:38.117644 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd8835e-182e-4978-9576-a32ea4cc75c1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:38.122375 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed4799-e077-4897-9f3e-566b2366c3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:38.122988 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed4799-e077-4897-9f3e-566b2366c3ad admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:38.123269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed4799-e077-4897-9f3e-566b2366c3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:38.123541 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed4799-e077-4897-9f3e-566b2366c3ad admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:38.124157 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed4799-e077-4897-9f3e-566b2366c3ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:38.126067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed4799-e077-4897-9f3e-566b2366c3ad admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:38.284666 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1401/2822] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:23:37 2022] POST /identity/v3/users => generated 297 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:23:38.302445 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafde145-37a2-4d88-9ac3-b0fa5de7add3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:38.312988 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafde145-37a2-4d88-9ac3-b0fa5de7add3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:38.318253 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c8c36-bba2-4415-b9a8-d36e9cde547f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:38.318874 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c8c36-bba2-4415-b9a8-d36e9cde547f admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:38.319146 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c8c36-bba2-4415-b9a8-d36e9cde547f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:38.319411 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c8c36-bba2-4415-b9a8-d36e9cde547f admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:38.320048 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c8c36-bba2-4415-b9a8-d36e9cde547f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:38.321632 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c8c36-bba2-4415-b9a8-d36e9cde547f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:38.327017 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:23:38.332701 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1402/2823] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:23:38.336875 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c8c36-bba2-4415-b9a8-d36e9cde547f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:38.347539 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c8c36-bba2-4415-b9a8-d36e9cde547f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:38.352370 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ec84a-4a6c-4e25-9d3e-f6b7aa7edfd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:38.353030 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ec84a-4a6c-4e25-9d3e-f6b7aa7edfd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:38.353308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ec84a-4a6c-4e25-9d3e-f6b7aa7edfd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:38.353576 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ec84a-4a6c-4e25-9d3e-f6b7aa7edfd8 admin admin] PATH_INFO: `/v3/projects/7116cdeb8c5f4e6ab05a65cd52f41e1d/users/e61719595bda495ca47aa9012bb8d2f6/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:38.369552 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ec84a-4a6c-4e25-9d3e-f6b7aa7edfd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7116cdeb8c5f4e6ab05a65cd52f41e1d, user_id=e61719595bda495ca47aa9012bb8d2f6, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:38.372282 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ec84a-4a6c-4e25-9d3e-f6b7aa7edfd8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:38.403421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1422/2824] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:23:38 2022] POST /identity/v3/users => generated 298 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:23:38.407423 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed4799-e077-4897-9f3e-566b2366c3ad admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:38.417451 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed4799-e077-4897-9f3e-566b2366c3ad admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:38.422164 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d0c64-94f5-4fdf-9ff2-a40fc0270808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:38.422810 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d0c64-94f5-4fdf-9ff2-a40fc0270808 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:38.423120 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d0c64-94f5-4fdf-9ff2-a40fc0270808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:38.424067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d0c64-94f5-4fdf-9ff2-a40fc0270808 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:38.424704 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d0c64-94f5-4fdf-9ff2-a40fc0270808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:38.426295 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d0c64-94f5-4fdf-9ff2-a40fc0270808 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:38.436586 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1423/2825] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:23:38.441470 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d0c64-94f5-4fdf-9ff2-a40fc0270808 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:38.452414 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d0c64-94f5-4fdf-9ff2-a40fc0270808 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:38.456273 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617a08a-151d-4be1-850d-7f3629fa129a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:38.456873 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617a08a-151d-4be1-850d-7f3629fa129a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:38.457150 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617a08a-151d-4be1-850d-7f3629fa129a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:38.457421 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617a08a-151d-4be1-850d-7f3629fa129a admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:38.458127 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617a08a-151d-4be1-850d-7f3629fa129a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:38.459774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617a08a-151d-4be1-850d-7f3629fa129a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:38.471845 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1424/2826] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:23:38.476065 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617a08a-151d-4be1-850d-7f3629fa129a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:38.481708 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1403/2827] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:23:38 2022] PUT /identity/v3/projects/7116cdeb8c5f4e6ab05a65cd52f41e1d/users/e61719595bda495ca47aa9012bb8d2f6/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:23:38.485288 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ec84a-4a6c-4e25-9d3e-f6b7aa7edfd8 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:38.487498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617a08a-151d-4be1-850d-7f3629fa129a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:38.553580 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6631e-dd30-457f-ada5-0e81352ca376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:38.554241 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6631e-dd30-457f-ada5-0e81352ca376 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:38.554588 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6631e-dd30-457f-ada5-0e81352ca376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:38.554994 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6631e-dd30-457f-ada5-0e81352ca376 admin admin] PATH_INFO: `/v3/projects/0d7130b2a1774b0b9007f9f503fce880/users/9dc07b6c0328494c9d541b759637932c/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:38.556702 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ec84a-4a6c-4e25-9d3e-f6b7aa7edfd8 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:38.562236 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4245c-54a7-4f11-91c3-6b917fc6fc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:38.563016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4245c-54a7-4f11-91c3-6b917fc6fc41 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:38.563368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4245c-54a7-4f11-91c3-6b917fc6fc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:38.563733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4245c-54a7-4f11-91c3-6b917fc6fc41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:38.564516 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df4245c-54a7-4f11-91c3-6b917fc6fc41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:38.566263 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df4245c-54a7-4f11-91c3-6b917fc6fc41 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:38.570820 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db6631e-dd30-457f-ada5-0e81352ca376 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d7130b2a1774b0b9007f9f503fce880, user_id=9dc07b6c0328494c9d541b759637932c, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:38.573817 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db6631e-dd30-457f-ada5-0e81352ca376 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:38.580235 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1404/2828] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:23:38.581281 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4245c-54a7-4f11-91c3-6b917fc6fc41 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:38.591465 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4245c-54a7-4f11-91c3-6b917fc6fc41 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:38.596958 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a004bb-a97e-4645-91ae-507ae800ea6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:38.597644 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a004bb-a97e-4645-91ae-507ae800ea6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:38.597917 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a004bb-a97e-4645-91ae-507ae800ea6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:38.598185 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a004bb-a97e-4645-91ae-507ae800ea6d admin admin] PATH_INFO: `/v3/projects/7116cdeb8c5f4e6ab05a65cd52f41e1d/users/e61719595bda495ca47aa9012bb8d2f6/roles/7554b353873942fab506de830188aba0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:38.600271 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a004bb-a97e-4645-91ae-507ae800ea6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7116cdeb8c5f4e6ab05a65cd52f41e1d, user_id=e61719595bda495ca47aa9012bb8d2f6, role_id=7554b353873942fab506de830188aba0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:38.603033 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a004bb-a97e-4645-91ae-507ae800ea6d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:38.615316 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1405/2829] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:23:38 2022] PUT /identity/v3/projects/7116cdeb8c5f4e6ab05a65cd52f41e1d/users/e61719595bda495ca47aa9012bb8d2f6/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:23:38.774702 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:23:38.780930 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1425/2830] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:23:38 2022] PUT /identity/v3/projects/0d7130b2a1774b0b9007f9f503fce880/users/9dc07b6c0328494c9d541b759637932c/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:23:38.792629 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be51417-1d5c-4b1a-9114-ca6d3642c620 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:38.792916 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be51417-1d5c-4b1a-9114-ca6d3642c620 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:38.793193 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be51417-1d5c-4b1a-9114-ca6d3642c620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:38.824004 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2be51417-1d5c-4b1a-9114-ca6d3642c620 None None] MFA Rules not processed for user `9dc07b6c0328494c9d541b759637932c`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:23:38.825302 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2be51417-1d5c-4b1a-9114-ca6d3642c620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:38.905242 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1406/2831] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:23:38 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:23:38.916967 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6631e-dd30-457f-ada5-0e81352ca376 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:38.918727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0db6631e-dd30-457f-ada5-0e81352ca376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:38.977006 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6631e-dd30-457f-ada5-0e81352ca376 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:38.980694 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ae6b1-12c2-43f5-86d4-a573fe502ed1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56d5c4afb9844fbbbe699f958deae6e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:38.981296 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ae6b1-12c2-43f5-86d4-a573fe502ed1 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:38.981573 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ae6b1-12c2-43f5-86d4-a573fe502ed1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:38.981840 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ae6b1-12c2-43f5-86d4-a573fe502ed1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:38.991281 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ae6b1-12c2-43f5-86d4-a573fe502ed1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:38.992759 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ae6b1-12c2-43f5-86d4-a573fe502ed1 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:39.011064 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1426/2832] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:23:38 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:23:44.922527 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb974caf-9696-46b2-ab11-fa9cd50da004 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:44.923049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb974caf-9696-46b2-ab11-fa9cd50da004 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:44.923049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb974caf-9696-46b2-ab11-fa9cd50da004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:44.952972 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-cb974caf-9696-46b2-ab11-fa9cd50da004 None None] MFA Rules not processed for user `e61719595bda495ca47aa9012bb8d2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:23:44.954211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cb974caf-9696-46b2-ab11-fa9cd50da004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:45.042978 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:23:45.102927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1407/2833] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:23:44 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:23:45.118711 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ae6b1-12c2-43f5-86d4-a573fe502ed1 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:45.120177 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-907ae6b1-12c2-43f5-86d4-a573fe502ed1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:45.183690 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ae6b1-12c2-43f5-86d4-a573fe502ed1 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:45.189373 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df44c5-8030-47ac-93f4-9ea47d2e1ddb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10f4f54890624da29ff217081445b6bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:45.190130 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df44c5-8030-47ac-93f4-9ea47d2e1ddb service glance] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:45.190500 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df44c5-8030-47ac-93f4-9ea47d2e1ddb service glance] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:45.190940 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df44c5-8030-47ac-93f4-9ea47d2e1ddb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:45.192786 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a3df44c5-8030-47ac-93f4-9ea47d2e1ddb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:45.257347 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3df44c5-8030-47ac-93f4-9ea47d2e1ddb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:45.258851 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3df44c5-8030-47ac-93f4-9ea47d2e1ddb service glance] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:45.270890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1427/2834] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:23:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:23:45.596812 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d0386-33b8-461c-8b8f-0eba6b782344 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:45.597094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d0386-33b8-461c-8b8f-0eba6b782344 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:45.597365 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d0386-33b8-461c-8b8f-0eba6b782344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:45.632989 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-014d0386-33b8-461c-8b8f-0eba6b782344 None None] MFA Rules not processed for user `4f9e1767ec90435f80248a2930081265`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:23:45.634264 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-014d0386-33b8-461c-8b8f-0eba6b782344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:45.697039 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1408/2835] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 10 17:23:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:23:45.890229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df44c5-8030-47ac-93f4-9ea47d2e1ddb service glance] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:45.890229 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a3df44c5-8030-47ac-93f4-9ea47d2e1ddb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:45.945462 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:23:45.964404 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df44c5-8030-47ac-93f4-9ea47d2e1ddb service glance] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:45.971222 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f5624-3a0e-4147-a23b-8ed81f5121fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4986821ef9f84b99a835e387635aa9a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:45.971917 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f5624-3a0e-4147-a23b-8ed81f5121fa service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:45.972417 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f5624-3a0e-4147-a23b-8ed81f5121fa service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:45.972515 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f5624-3a0e-4147-a23b-8ed81f5121fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:45.991015 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f5624-3a0e-4147-a23b-8ed81f5121fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:45.991015 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f5624-3a0e-4147-a23b-8ed81f5121fa service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:45.993830 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1428/2836] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 10 17:23:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 17:23:48.585347 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d0386-33b8-461c-8b8f-0eba6b782344 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:48.691948 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d0386-33b8-461c-8b8f-0eba6b782344 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:48.698004 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26fae7-7a24-4bcb-93ae-2b79f1014e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:48.702344 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26fae7-7a24-4bcb-93ae-2b79f1014e18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:48.707547 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26fae7-7a24-4bcb-93ae-2b79f1014e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:48.707832 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26fae7-7a24-4bcb-93ae-2b79f1014e18 admin admin] PATH_INFO: `/v3/users/9dc07b6c0328494c9d541b759637932c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:48.709456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b26fae7-7a24-4bcb-93ae-2b79f1014e18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dc07b6c0328494c9d541b759637932c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:48.713751 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b26fae7-7a24-4bcb-93ae-2b79f1014e18 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:49.231719 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6b26fae7-7a24-4bcb-93ae-2b79f1014e18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dc07b6c0328494c9d541b759637932c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:49.476335 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6b26fae7-7a24-4bcb-93ae-2b79f1014e18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dc07b6c0328494c9d541b759637932c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:49.490136 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:23:49.495203 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6b26fae7-7a24-4bcb-93ae-2b79f1014e18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dc07b6c0328494c9d541b759637932c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:49.496061 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6b26fae7-7a24-4bcb-93ae-2b79f1014e18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dc07b6c0328494c9d541b759637932c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:49.497036 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6b26fae7-7a24-4bcb-93ae-2b79f1014e18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dc07b6c0328494c9d541b759637932c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:49.514164 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1409/2837] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:23:48 2022] DELETE /identity/v3/users/9dc07b6c0328494c9d541b759637932c => generated 0 bytes in 929 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:23:49.521442 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977efe7-4208-450d-81b7-375b6d16a8ac None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:49.521975 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977efe7-4208-450d-81b7-375b6d16a8ac None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:49.522090 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977efe7-4208-450d-81b7-375b6d16a8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:49.542291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7977efe7-4208-450d-81b7-375b6d16a8ac None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:23:49.543944 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7977efe7-4208-450d-81b7-375b6d16a8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:49.623830 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1429/2838] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:23:49.633779 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26fae7-7a24-4bcb-93ae-2b79f1014e18 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:49.635367 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6b26fae7-7a24-4bcb-93ae-2b79f1014e18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:49.706951 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26fae7-7a24-4bcb-93ae-2b79f1014e18 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:49.711162 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637cd5aa-3418-4065-b58f-172527afbbf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:49.711905 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cd5aa-3418-4065-b58f-172527afbbf0 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:49.712278 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cd5aa-3418-4065-b58f-172527afbbf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:49.712658 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cd5aa-3418-4065-b58f-172527afbbf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:49.725685 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637cd5aa-3418-4065-b58f-172527afbbf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:49.727523 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637cd5aa-3418-4065-b58f-172527afbbf0 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:49.740810 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1410/2839] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:23:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:23:50.250086 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7977efe7-4208-450d-81b7-375b6d16a8ac None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:23:50.256808 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7977efe7-4208-450d-81b7-375b6d16a8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:50.268803 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7977efe7-4208-450d-81b7-375b6d16a8ac None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:23:50.280785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe3199-3556-4113-86a4-ee3e2c8d9da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:23:50.286624 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe3199-3556-4113-86a4-ee3e2c8d9da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:50.286624 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe3199-3556-4113-86a4-ee3e2c8d9da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:50.286624 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe3199-3556-4113-86a4-ee3e2c8d9da9 admin admin] PATH_INFO: `/v3/projects/0d7130b2a1774b0b9007f9f503fce880` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:50.286624 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe3199-3556-4113-86a4-ee3e2c8d9da9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d7130b2a1774b0b9007f9f503fce880)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:23:50.287404 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe3199-3556-4113-86a4-ee3e2c8d9da9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:23:50.409877 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4fbe3199-3556-4113-86a4-ee3e2c8d9da9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d7130b2a1774b0b9007f9f503fce880', 'request_id': 'req-4fbe3199-3556-4113-86a4-ee3e2c8d9da9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:50.418194 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4fbe3199-3556-4113-86a4-ee3e2c8d9da9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d7130b2a1774b0b9007f9f503fce880', 'request_id': 'req-4fbe3199-3556-4113-86a4-ee3e2c8d9da9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:50.527726 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4fbe3199-3556-4113-86a4-ee3e2c8d9da9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d7130b2a1774b0b9007f9f503fce880', 'request_id': 'req-4fbe3199-3556-4113-86a4-ee3e2c8d9da9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:50.540475 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4fbe3199-3556-4113-86a4-ee3e2c8d9da9 admin admin] The token cache is being invalidate because project 0d7130b2a1774b0b9007f9f503fce880 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:23:50.542614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4fbe3199-3556-4113-86a4-ee3e2c8d9da9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:50.542614 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4fbe3199-3556-4113-86a4-ee3e2c8d9da9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:23:50.543443 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1430/2840] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:23:50 2022] DELETE /identity/v3/projects/0d7130b2a1774b0b9007f9f503fce880 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:23:56.161544 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ee65e-955e-4067-913b-1eb9e06d65fd None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:23:56.162045 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ee65e-955e-4067-913b-1eb9e06d65fd None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:23:56.162148 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ee65e-955e-4067-913b-1eb9e06d65fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:23:56.181777 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ad5ee65e-955e-4067-913b-1eb9e06d65fd None None] MFA Rules not processed for user `dd1ac5036e16409a9a5cac62fed7d7e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:23:56.183014 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ad5ee65e-955e-4067-913b-1eb9e06d65fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:23:56.237848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1411/2841] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:23:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 17:24:00.537395 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=61,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:24:04.844592 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe3199-3556-4113-86a4-ee3e2c8d9da9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:24:04.846110 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4fbe3199-3556-4113-86a4-ee3e2c8d9da9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:24:04.903679 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe3199-3556-4113-86a4-ee3e2c8d9da9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:24:04.907292 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cac0b3-ee3b-459d-b6c5-628324b7bc2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:24:04.907891 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cac0b3-ee3b-459d-b6c5-628324b7bc2d service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:24:04.908172 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cac0b3-ee3b-459d-b6c5-628324b7bc2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:24:04.908442 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cac0b3-ee3b-459d-b6c5-628324b7bc2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:24:04.910242 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-27cac0b3-ee3b-459d-b6c5-628324b7bc2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:24:05.122706 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cac0b3-ee3b-459d-b6c5-628324b7bc2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:24:05.124137 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cac0b3-ee3b-459d-b6c5-628324b7bc2d service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:24:05.145537 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1431/2842] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:24:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:24:06.230140 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=58,keystone:DELETE=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:24:06.256376 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ee65e-955e-4067-913b-1eb9e06d65fd None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:24:06.266656 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ee65e-955e-4067-913b-1eb9e06d65fd None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:24:06.270517 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51161f24-24ec-4de0-b519-84f80e1fa013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:24:06.272235 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51161f24-24ec-4de0-b519-84f80e1fa013 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:24:06.272235 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51161f24-24ec-4de0-b519-84f80e1fa013 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:24:06.272235 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51161f24-24ec-4de0-b519-84f80e1fa013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:24:06.273765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-51161f24-24ec-4de0-b519-84f80e1fa013 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:24:06.289185 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51161f24-24ec-4de0-b519-84f80e1fa013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:24:06.290684 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51161f24-24ec-4de0-b519-84f80e1fa013 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:24:06.308790 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1412/2843] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:24:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:24:06.707834 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7777e-d088-44bd-a984-d2b0309a264a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:24:06.708330 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7777e-d088-44bd-a984-d2b0309a264a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:24:06.708540 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7777e-d088-44bd-a984-d2b0309a264a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:24:06.727883 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5ab7777e-d088-44bd-a984-d2b0309a264a None None] MFA Rules not processed for user `4f9e1767ec90435f80248a2930081265`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:24:06.729115 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5ab7777e-d088-44bd-a984-d2b0309a264a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:24:06.792735 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1432/2844] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 10 17:24:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:24:06.805231 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51161f24-24ec-4de0-b519-84f80e1fa013 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:24:06.807289 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-51161f24-24ec-4de0-b519-84f80e1fa013 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:24:06.878698 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51161f24-24ec-4de0-b519-84f80e1fa013 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:24:06.884119 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7088f15-4048-4260-a83e-bd853c2d6f37 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4986821ef9f84b99a835e387635aa9a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:24:06.884737 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7088f15-4048-4260-a83e-bd853c2d6f37 service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:24:06.885579 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7088f15-4048-4260-a83e-bd853c2d6f37 service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:24:06.887583 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7088f15-4048-4260-a83e-bd853c2d6f37 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:24:06.894074 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7088f15-4048-4260-a83e-bd853c2d6f37 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:24:06.895709 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7088f15-4048-4260-a83e-bd853c2d6f37 service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:24:06.904278 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1413/2845] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 10 17:24:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 17:24:16.785105 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=88 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:24:16.901368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=38 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:24:33.485127 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b763e0-75c5-49ed-87c6-a99c23f33836 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:24:33.485701 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b763e0-75c5-49ed-87c6-a99c23f33836 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:24:33.485701 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b763e0-75c5-49ed-87c6-a99c23f33836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:24:33.505451 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-79b763e0-75c5-49ed-87c6-a99c23f33836 None None] MFA Rules not processed for user `0d7d26ea8e26466f91b1905aa1e2e156`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:24:33.506878 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-79b763e0-75c5-49ed-87c6-a99c23f33836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:24:33.567233 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1433/2846] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:24:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:24:37.964084 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7088f15-4048-4260-a83e-bd853c2d6f37 service swift] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:24:37.974797 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7088f15-4048-4260-a83e-bd853c2d6f37 service swift] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:24:37.978522 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91916e74-0c2f-4c6d-88e4-2e1e705427e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:24:37.979211 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91916e74-0c2f-4c6d-88e4-2e1e705427e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:24:37.979490 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91916e74-0c2f-4c6d-88e4-2e1e705427e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:24:37.985149 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91916e74-0c2f-4c6d-88e4-2e1e705427e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:24:37.985149 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-91916e74-0c2f-4c6d-88e4-2e1e705427e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:24:38.051228 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91916e74-0c2f-4c6d-88e4-2e1e705427e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:24:38.052671 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91916e74-0c2f-4c6d-88e4-2e1e705427e4 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:24:38.063267 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1414/2847] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:24:37 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:24:43.559428 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=25 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:24:48.058247 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=30 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:25:13.819356 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b763e0-75c5-49ed-87c6-a99c23f33836 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:25:13.821066 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-79b763e0-75c5-49ed-87c6-a99c23f33836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:25:13.836380 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b763e0-75c5-49ed-87c6-a99c23f33836 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:25:13.840604 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fdfd3a-4763-4b0e-b9e1-947c54e7a1e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:25:13.841353 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdfd3a-4763-4b0e-b9e1-947c54e7a1e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:25:13.841633 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdfd3a-4763-4b0e-b9e1-947c54e7a1e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:25:13.841903 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdfd3a-4763-4b0e-b9e1-947c54e7a1e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:25:13.843764 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a6fdfd3a-4763-4b0e-b9e1-947c54e7a1e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:25:13.853984 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fdfd3a-4763-4b0e-b9e1-947c54e7a1e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:25:13.855452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fdfd3a-4763-4b0e-b9e1-947c54e7a1e3 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:25:13.865697 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1434/2848] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:25:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:25:14.282098 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91916e74-0c2f-4c6d-88e4-2e1e705427e4 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:25:14.283602 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-91916e74-0c2f-4c6d-88e4-2e1e705427e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:25:14.299921 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91916e74-0c2f-4c6d-88e4-2e1e705427e4 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:25:14.305432 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6af74-f25a-4fe9-bbb4-eb0ab239809b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:25:14.306038 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6af74-f25a-4fe9-bbb4-eb0ab239809b service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:25:14.306311 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6af74-f25a-4fe9-bbb4-eb0ab239809b service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:25:14.306612 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6af74-f25a-4fe9-bbb4-eb0ab239809b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:25:14.308415 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-28f6af74-f25a-4fe9-bbb4-eb0ab239809b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:25:14.321065 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f6af74-f25a-4fe9-bbb4-eb0ab239809b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:25:14.322676 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f6af74-f25a-4fe9-bbb4-eb0ab239809b service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:25:14.333400 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1415/2849] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:25:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:25:23.860526 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:25:24.328824 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:25:43.030943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fdfd3a-4763-4b0e-b9e1-947c54e7a1e3 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:25:43.036548 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a6fdfd3a-4763-4b0e-b9e1-947c54e7a1e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:25:43.098107 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fdfd3a-4763-4b0e-b9e1-947c54e7a1e3 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:25:43.102483 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ececea-6e4d-4a08-ab82-83c24b57aa68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:25:43.103098 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ececea-6e4d-4a08-ab82-83c24b57aa68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:25:43.103369 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ececea-6e4d-4a08-ab82-83c24b57aa68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:25:43.103633 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ececea-6e4d-4a08-ab82-83c24b57aa68 admin admin] PATH_INFO: `/v3/users/0d7d26ea8e26466f91b1905aa1e2e156` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:25:43.104752 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ececea-6e4d-4a08-ab82-83c24b57aa68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d7d26ea8e26466f91b1905aa1e2e156)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:25:43.106664 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ececea-6e4d-4a08-ab82-83c24b57aa68 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:25:43.298635 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-02ececea-6e4d-4a08-ab82-83c24b57aa68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d7d26ea8e26466f91b1905aa1e2e156'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:25:43.382881 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-02ececea-6e4d-4a08-ab82-83c24b57aa68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d7d26ea8e26466f91b1905aa1e2e156'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:25:43.390627 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-02ececea-6e4d-4a08-ab82-83c24b57aa68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d7d26ea8e26466f91b1905aa1e2e156'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:25:43.402483 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-02ececea-6e4d-4a08-ab82-83c24b57aa68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d7d26ea8e26466f91b1905aa1e2e156'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:25:43.403201 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-02ececea-6e4d-4a08-ab82-83c24b57aa68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d7d26ea8e26466f91b1905aa1e2e156'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:25:43.408987 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1435/2850] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:25:43 2022] DELETE /identity/v3/users/0d7d26ea8e26466f91b1905aa1e2e156 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:25:43.659076 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6af74-f25a-4fe9-bbb4-eb0ab239809b service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:25:43.669995 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6af74-f25a-4fe9-bbb4-eb0ab239809b service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:25:43.735554 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7a43d-dc7e-492d-8a22-e07cbe6bd953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:25:43.736232 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7a43d-dc7e-492d-8a22-e07cbe6bd953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:25:43.736511 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7a43d-dc7e-492d-8a22-e07cbe6bd953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:25:43.736813 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7a43d-dc7e-492d-8a22-e07cbe6bd953 admin admin] PATH_INFO: `/v3/projects/a49a6fc2c18d44b8a8b82307021355fd` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:25:43.738600 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e7a43d-dc7e-492d-8a22-e07cbe6bd953 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a49a6fc2c18d44b8a8b82307021355fd)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:25:43.740564 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e7a43d-dc7e-492d-8a22-e07cbe6bd953 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:25:43.872323 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-73e7a43d-dc7e-492d-8a22-e07cbe6bd953 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a49a6fc2c18d44b8a8b82307021355fd', 'request_id': 'req-73e7a43d-dc7e-492d-8a22-e07cbe6bd953'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:25:43.942587 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-73e7a43d-dc7e-492d-8a22-e07cbe6bd953 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a49a6fc2c18d44b8a8b82307021355fd', 'request_id': 'req-73e7a43d-dc7e-492d-8a22-e07cbe6bd953'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:25:43.949423 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-73e7a43d-dc7e-492d-8a22-e07cbe6bd953 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a49a6fc2c18d44b8a8b82307021355fd', 'request_id': 'req-73e7a43d-dc7e-492d-8a22-e07cbe6bd953'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:25:43.960368 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-73e7a43d-dc7e-492d-8a22-e07cbe6bd953 admin admin] The token cache is being invalidate because project a49a6fc2c18d44b8a8b82307021355fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:25:43.964716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-73e7a43d-dc7e-492d-8a22-e07cbe6bd953 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:25:43.964716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-73e7a43d-dc7e-492d-8a22-e07cbe6bd953 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:25:43.964716 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1416/2851] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:25:43 2022] DELETE /identity/v3/projects/a49a6fc2c18d44b8a8b82307021355fd => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:25:53.405575 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:25:53.957186 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:26:03.951570 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ececea-6e4d-4a08-ab82-83c24b57aa68 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:26:03.952745 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-02ececea-6e4d-4a08-ab82-83c24b57aa68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:26:04.059900 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ececea-6e4d-4a08-ab82-83c24b57aa68 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:26:04.143411 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb5d1b-bbfc-4c42-8ebd-625ed18f4995 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:26:04.144440 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb5d1b-bbfc-4c42-8ebd-625ed18f4995 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:04.144651 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb5d1b-bbfc-4c42-8ebd-625ed18f4995 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:04.144880 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb5d1b-bbfc-4c42-8ebd-625ed18f4995 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:04.146920 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1efb5d1b-bbfc-4c42-8ebd-625ed18f4995 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:26:04.159671 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb5d1b-bbfc-4c42-8ebd-625ed18f4995 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:26:04.167435 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb5d1b-bbfc-4c42-8ebd-625ed18f4995 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:26:04.173946 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1436/2852] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:26:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:26:13.136247 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ddd10-e9bb-4470-9322-64429ce145d9 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:13.136713 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ddd10-e9bb-4470-9322-64429ce145d9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:13.136817 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ddd10-e9bb-4470-9322-64429ce145d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:13.192070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f45ddd10-e9bb-4470-9322-64429ce145d9 None None] MFA Rules not processed for user `4f9e1767ec90435f80248a2930081265`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:26:13.192070 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f45ddd10-e9bb-4470-9322-64429ce145d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:26:13.251843 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1417/2853] 10.222.0.45 () {58 vars in 967 bytes} [Wed Aug 10 17:26:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:26:13.296407 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb5d1b-bbfc-4c42-8ebd-625ed18f4995 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:26:13.297784 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1efb5d1b-bbfc-4c42-8ebd-625ed18f4995 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:26:13.387333 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb5d1b-bbfc-4c42-8ebd-625ed18f4995 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:26:13.392397 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9c6357-705e-4d4b-aaf9-28f61859d2e2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4986821ef9f84b99a835e387635aa9a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:26:13.392996 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c6357-705e-4d4b-aaf9-28f61859d2e2 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:13.393273 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c6357-705e-4d4b-aaf9-28f61859d2e2 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:13.393545 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c6357-705e-4d4b-aaf9-28f61859d2e2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:13.398710 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:26:13.402333 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9c6357-705e-4d4b-aaf9-28f61859d2e2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:26:13.403952 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9c6357-705e-4d4b-aaf9-28f61859d2e2 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:26:13.411943 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1437/2854] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 10 17:26:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 17:26:21.801854 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ddd10-e9bb-4470-9322-64429ce145d9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:26:21.803525 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f45ddd10-e9bb-4470-9322-64429ce145d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:26:21.887932 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ddd10-e9bb-4470-9322-64429ce145d9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:26:21.893343 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8469a-053a-4fd2-80ff-44ec9b6943d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cd39d9a583a4d73a5654017be106019', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:26:21.893965 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8469a-053a-4fd2-80ff-44ec9b6943d7 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:21.894256 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8469a-053a-4fd2-80ff-44ec9b6943d7 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:21.894557 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8469a-053a-4fd2-80ff-44ec9b6943d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:21.896537 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-32f8469a-053a-4fd2-80ff-44ec9b6943d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:26:21.909161 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8469a-053a-4fd2-80ff-44ec9b6943d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:26:21.912837 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8469a-053a-4fd2-80ff-44ec9b6943d7 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:26:21.922653 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1418/2855] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:26:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:26:21.998525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9c6357-705e-4d4b-aaf9-28f61859d2e2 service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:26:21.998525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4f9c6357-705e-4d4b-aaf9-28f61859d2e2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:26:22.068510 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9c6357-705e-4d4b-aaf9-28f61859d2e2 service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:26:22.073401 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a52b1-e75b-4b0a-8bf7-4b8cbc126006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:26:22.074040 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a52b1-e75b-4b0a-8bf7-4b8cbc126006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:22.074362 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a52b1-e75b-4b0a-8bf7-4b8cbc126006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:22.074646 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a52b1-e75b-4b0a-8bf7-4b8cbc126006 admin admin] PATH_INFO: `/v3/users/dd1ac5036e16409a9a5cac62fed7d7e7` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:22.075781 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a52b1-e75b-4b0a-8bf7-4b8cbc126006 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd1ac5036e16409a9a5cac62fed7d7e7)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:26:22.077730 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a52b1-e75b-4b0a-8bf7-4b8cbc126006 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:26:22.266269 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-409a52b1-e75b-4b0a-8bf7-4b8cbc126006 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd1ac5036e16409a9a5cac62fed7d7e7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:26:22.382785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-409a52b1-e75b-4b0a-8bf7-4b8cbc126006 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd1ac5036e16409a9a5cac62fed7d7e7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:26:22.390890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-409a52b1-e75b-4b0a-8bf7-4b8cbc126006 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd1ac5036e16409a9a5cac62fed7d7e7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:26:22.403186 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-409a52b1-e75b-4b0a-8bf7-4b8cbc126006 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd1ac5036e16409a9a5cac62fed7d7e7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:26:22.403894 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-409a52b1-e75b-4b0a-8bf7-4b8cbc126006 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd1ac5036e16409a9a5cac62fed7d7e7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:26:22.412323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1438/2856] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:26:21 2022] DELETE /identity/v3/users/dd1ac5036e16409a9a5cac62fed7d7e7 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:26:22.685746 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8469a-053a-4fd2-80ff-44ec9b6943d7 service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:26:22.697348 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8469a-053a-4fd2-80ff-44ec9b6943d7 service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:26:22.765515 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b36e0-01d2-4ac8-8786-186e9d82facc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:26:22.766153 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b36e0-01d2-4ac8-8786-186e9d82facc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:22.766456 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b36e0-01d2-4ac8-8786-186e9d82facc admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:22.766732 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b36e0-01d2-4ac8-8786-186e9d82facc admin admin] PATH_INFO: `/v3/projects/0ca8a0c2eb86470cb9e609f05e4ceee7` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:22.768084 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b36e0-01d2-4ac8-8786-186e9d82facc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ca8a0c2eb86470cb9e609f05e4ceee7)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:26:22.770846 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b36e0-01d2-4ac8-8786-186e9d82facc admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:26:22.773588 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:26:22.881255 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a07b36e0-01d2-4ac8-8786-186e9d82facc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ca8a0c2eb86470cb9e609f05e4ceee7', 'request_id': 'req-a07b36e0-01d2-4ac8-8786-186e9d82facc'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:26:22.970799 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a07b36e0-01d2-4ac8-8786-186e9d82facc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ca8a0c2eb86470cb9e609f05e4ceee7', 'request_id': 'req-a07b36e0-01d2-4ac8-8786-186e9d82facc'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:26:22.978664 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a07b36e0-01d2-4ac8-8786-186e9d82facc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ca8a0c2eb86470cb9e609f05e4ceee7', 'request_id': 'req-a07b36e0-01d2-4ac8-8786-186e9d82facc'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:26:22.991044 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a07b36e0-01d2-4ac8-8786-186e9d82facc admin admin] The token cache is being invalidate because project 0ca8a0c2eb86470cb9e609f05e4ceee7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:26:22.991340 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a07b36e0-01d2-4ac8-8786-186e9d82facc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:26:22.993350 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a07b36e0-01d2-4ac8-8786-186e9d82facc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:26:22.995224 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1419/2857] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:26:22 2022] DELETE /identity/v3/projects/0ca8a0c2eb86470cb9e609f05e4ceee7 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:26:23.001138 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a52b1-e75b-4b0a-8bf7-4b8cbc126006 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:26:23.003423 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-409a52b1-e75b-4b0a-8bf7-4b8cbc126006 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:26:23.018516 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a52b1-e75b-4b0a-8bf7-4b8cbc126006 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:26:23.084883 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414fc95-659e-4528-a692-ae03abe2cbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:26:23.086281 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414fc95-659e-4528-a692-ae03abe2cbc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:23.086584 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414fc95-659e-4528-a692-ae03abe2cbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:23.086853 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414fc95-659e-4528-a692-ae03abe2cbc7 admin admin] PATH_INFO: `/v3/users/e61719595bda495ca47aa9012bb8d2f6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:23.088018 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414fc95-659e-4528-a692-ae03abe2cbc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e61719595bda495ca47aa9012bb8d2f6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:26:23.089917 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414fc95-659e-4528-a692-ae03abe2cbc7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:26:23.105457 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:26:23.294232 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2414fc95-659e-4528-a692-ae03abe2cbc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e61719595bda495ca47aa9012bb8d2f6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:26:23.348921 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2414fc95-659e-4528-a692-ae03abe2cbc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e61719595bda495ca47aa9012bb8d2f6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:26:23.357423 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2414fc95-659e-4528-a692-ae03abe2cbc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e61719595bda495ca47aa9012bb8d2f6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:26:23.373641 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2414fc95-659e-4528-a692-ae03abe2cbc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e61719595bda495ca47aa9012bb8d2f6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:26:23.374828 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2414fc95-659e-4528-a692-ae03abe2cbc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e61719595bda495ca47aa9012bb8d2f6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:26:23.384313 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1439/2858] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:26:22 2022] DELETE /identity/v3/users/e61719595bda495ca47aa9012bb8d2f6 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:26:23.700410 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b36e0-01d2-4ac8-8786-186e9d82facc admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:26:23.702261 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a07b36e0-01d2-4ac8-8786-186e9d82facc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:26:23.723125 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b36e0-01d2-4ac8-8786-186e9d82facc admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:26:23.820842 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b8d1a-32b9-40d7-a9ad-ba98c9dbb138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:26:23.821795 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b8d1a-32b9-40d7-a9ad-ba98c9dbb138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:23.822068 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b8d1a-32b9-40d7-a9ad-ba98c9dbb138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:23.822343 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b8d1a-32b9-40d7-a9ad-ba98c9dbb138 admin admin] PATH_INFO: `/v3/projects/7116cdeb8c5f4e6ab05a65cd52f41e1d` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:23.823632 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b8d1a-32b9-40d7-a9ad-ba98c9dbb138 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7116cdeb8c5f4e6ab05a65cd52f41e1d)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:26:23.825571 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b8d1a-32b9-40d7-a9ad-ba98c9dbb138 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:26:23.949925 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-365b8d1a-32b9-40d7-a9ad-ba98c9dbb138 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7116cdeb8c5f4e6ab05a65cd52f41e1d', 'request_id': 'req-365b8d1a-32b9-40d7-a9ad-ba98c9dbb138'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:26:24.015858 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-365b8d1a-32b9-40d7-a9ad-ba98c9dbb138 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7116cdeb8c5f4e6ab05a65cd52f41e1d', 'request_id': 'req-365b8d1a-32b9-40d7-a9ad-ba98c9dbb138'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:26:24.023909 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-365b8d1a-32b9-40d7-a9ad-ba98c9dbb138 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7116cdeb8c5f4e6ab05a65cd52f41e1d', 'request_id': 'req-365b8d1a-32b9-40d7-a9ad-ba98c9dbb138'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:26:24.035298 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-365b8d1a-32b9-40d7-a9ad-ba98c9dbb138 admin admin] The token cache is being invalidate because project 7116cdeb8c5f4e6ab05a65cd52f41e1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:26:24.035298 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-365b8d1a-32b9-40d7-a9ad-ba98c9dbb138 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:26:24.036943 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-365b8d1a-32b9-40d7-a9ad-ba98c9dbb138 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:26:24.038493 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1420/2859] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:26:23 2022] DELETE /identity/v3/projects/7116cdeb8c5f4e6ab05a65cd52f41e1d => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 17:26:24.044452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4705b6-ede6-4765-b408-5cc3150c366e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:24.044452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4705b6-ede6-4765-b408-5cc3150c366e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:24.044452 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4705b6-ede6-4765-b408-5cc3150c366e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:24.070613 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0a4705b6-ede6-4765-b408-5cc3150c366e None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:26:24.071893 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0a4705b6-ede6-4765-b408-5cc3150c366e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:26:24.143489 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1440/2860] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:26:24.151556 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630529b-0974-4550-b846-80e20d5b978a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:24.151896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630529b-0974-4550-b846-80e20d5b978a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:24.152099 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630529b-0974-4550-b846-80e20d5b978a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:24.173208 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f630529b-0974-4550-b846-80e20d5b978a None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:26:24.174515 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f630529b-0974-4550-b846-80e20d5b978a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:26:24.183767 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1421/2861] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:26:24.188525 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4705b6-ede6-4765-b408-5cc3150c366e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:26:24.199704 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4705b6-ede6-4765-b408-5cc3150c366e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:26:24.205071 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28780e7a-c260-48ae-bf33-8d072889e639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:26:24.210837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28780e7a-c260-48ae-bf33-8d072889e639 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:24.210837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28780e7a-c260-48ae-bf33-8d072889e639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:24.210837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28780e7a-c260-48ae-bf33-8d072889e639 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:24.210837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28780e7a-c260-48ae-bf33-8d072889e639 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 17:26:24.210837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28780e7a-c260-48ae-bf33-8d072889e639 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:26:24.210837 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28780e7a-c260-48ae-bf33-8d072889e639 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:26:24.223773 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1441/2862] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 10 17:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 17:26:24.228524 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630529b-0974-4550-b846-80e20d5b978a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:26:24.239315 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630529b-0974-4550-b846-80e20d5b978a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:26:24.244271 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294988b7-6a7d-4374-b5f3-501c2f36f4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:26:24.244917 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294988b7-6a7d-4374-b5f3-501c2f36f4f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:24.245198 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294988b7-6a7d-4374-b5f3-501c2f36f4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:24.245464 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294988b7-6a7d-4374-b5f3-501c2f36f4f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:24.246098 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294988b7-6a7d-4374-b5f3-501c2f36f4f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:26:24.248069 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294988b7-6a7d-4374-b5f3-501c2f36f4f5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:26:24.266765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:26:24.356220 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1422/2863] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:26:24 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 17:26:24.361539 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28780e7a-c260-48ae-bf33-8d072889e639 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:26:24.385007 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28780e7a-c260-48ae-bf33-8d072889e639 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:26:24.454970 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e867d-e23a-4945-8bf4-73913da8c558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:26:24.455602 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e867d-e23a-4945-8bf4-73913da8c558 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:24.455886 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e867d-e23a-4945-8bf4-73913da8c558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:24.456130 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e867d-e23a-4945-8bf4-73913da8c558 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:24.456784 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e867d-e23a-4945-8bf4-73913da8c558 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:26:24.460227 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e867d-e23a-4945-8bf4-73913da8c558 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:26:24.548667 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1442/2864] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:26:24 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:26:24.553253 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294988b7-6a7d-4374-b5f3-501c2f36f4f5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:26:24.563861 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294988b7-6a7d-4374-b5f3-501c2f36f4f5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:26:24.568823 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc25cb2-8b5d-4156-8034-6907e5d1a205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:26:24.569440 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc25cb2-8b5d-4156-8034-6907e5d1a205 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:24.569708 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc25cb2-8b5d-4156-8034-6907e5d1a205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:24.569973 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc25cb2-8b5d-4156-8034-6907e5d1a205 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:24.570657 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc25cb2-8b5d-4156-8034-6907e5d1a205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:26:24.572264 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc25cb2-8b5d-4156-8034-6907e5d1a205 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:26:24.583831 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1423/2865] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:26:24.588002 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e867d-e23a-4945-8bf4-73913da8c558 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:26:24.600387 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e867d-e23a-4945-8bf4-73913da8c558 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:26:24.605785 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c5be0-e63e-443d-83ec-67ef8f0b455a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:26:24.606616 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c5be0-e63e-443d-83ec-67ef8f0b455a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:24.606901 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c5be0-e63e-443d-83ec-67ef8f0b455a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:24.607236 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c5be0-e63e-443d-83ec-67ef8f0b455a admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:24.607955 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c5be0-e63e-443d-83ec-67ef8f0b455a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:26:24.609793 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c5be0-e63e-443d-83ec-67ef8f0b455a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:26:24.612812 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:26:24.623336 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1443/2866] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:26:24.628287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc25cb2-8b5d-4156-8034-6907e5d1a205 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:26:24.640438 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc25cb2-8b5d-4156-8034-6907e5d1a205 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:26:24.646169 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d3548-10b6-4cbb-999e-a591316c2c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:26:24.647115 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d3548-10b6-4cbb-999e-a591316c2c4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:24.647504 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d3548-10b6-4cbb-999e-a591316c2c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:24.647878 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d3548-10b6-4cbb-999e-a591316c2c4d admin admin] PATH_INFO: `/v3/projects/c1f04e9a75b44271a9a2aa3834c5c9d4/users/a9736ec2b6c54177971db7277ca04c1d/roles/dfb72dc1f3744c8e957abcc4ac967dac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:24.667480 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d3548-10b6-4cbb-999e-a591316c2c4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1f04e9a75b44271a9a2aa3834c5c9d4, user_id=a9736ec2b6c54177971db7277ca04c1d, role_id=dfb72dc1f3744c8e957abcc4ac967dac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:26:24.670750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d3548-10b6-4cbb-999e-a591316c2c4d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:26:24.715670 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1424/2867] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:26:24 2022] PUT /identity/v3/projects/c1f04e9a75b44271a9a2aa3834c5c9d4/users/a9736ec2b6c54177971db7277ca04c1d/roles/dfb72dc1f3744c8e957abcc4ac967dac => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:26:24.720898 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310a5e0-2df1-4520-9562-835a07a769b5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:24.721182 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310a5e0-2df1-4520-9562-835a07a769b5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:24.721456 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310a5e0-2df1-4520-9562-835a07a769b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:24.741786 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7310a5e0-2df1-4520-9562-835a07a769b5 None None] MFA Rules not processed for user `164faaac1ec04c50ba5358bf58d221cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:26:24.743052 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7310a5e0-2df1-4520-9562-835a07a769b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:26:24.803928 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1444/2868] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:26:24.815000 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d3548-10b6-4cbb-999e-a591316c2c4d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:26:24.816789 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d01d3548-10b6-4cbb-999e-a591316c2c4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:26:24.876695 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d3548-10b6-4cbb-999e-a591316c2c4d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:26:24.881051 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37c3a7-5292-4700-9138-7bf252b68b0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:26:24.881660 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37c3a7-5292-4700-9138-7bf252b68b0f service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:24.881933 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37c3a7-5292-4700-9138-7bf252b68b0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:24.882203 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37c3a7-5292-4700-9138-7bf252b68b0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:24.891704 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa37c3a7-5292-4700-9138-7bf252b68b0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:26:24.893165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa37c3a7-5292-4700-9138-7bf252b68b0f service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:26:24.904037 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1425/2869] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:26:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:26:31.016323 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3cd70-c89f-42a2-a4ad-ed1649af9a47 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:31.016870 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3cd70-c89f-42a2-a4ad-ed1649af9a47 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:31.017199 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3cd70-c89f-42a2-a4ad-ed1649af9a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:31.068114 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9ff3cd70-c89f-42a2-a4ad-ed1649af9a47 None None] MFA Rules not processed for user `a9736ec2b6c54177971db7277ca04c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:26:31.070816 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9ff3cd70-c89f-42a2-a4ad-ed1649af9a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:26:31.168506 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1445/2870] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:26:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:26:31.174334 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37c3a7-5292-4700-9138-7bf252b68b0f service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:26:31.187994 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37c3a7-5292-4700-9138-7bf252b68b0f service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:26:31.194221 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ef133-4d91-43a3-93ff-0edf8fd6a2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:26:31.195620 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ef133-4d91-43a3-93ff-0edf8fd6a2e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:31.196012 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ef133-4d91-43a3-93ff-0edf8fd6a2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:31.196389 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ef133-4d91-43a3-93ff-0edf8fd6a2e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:31.197513 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ef133-4d91-43a3-93ff-0edf8fd6a2e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:26:31.200301 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ef133-4d91-43a3-93ff-0edf8fd6a2e4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:26:31.321202 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1426/2871] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 10 17:26:31 2022] POST /identity/v3/projects => generated 367 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:26:31.325718 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3cd70-c89f-42a2-a4ad-ed1649af9a47 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:26:31.337809 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3cd70-c89f-42a2-a4ad-ed1649af9a47 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:26:31.403876 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471cfd6b-89dc-4889-856e-1254ab42b719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:26:31.404495 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cfd6b-89dc-4889-856e-1254ab42b719 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:31.404770 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cfd6b-89dc-4889-856e-1254ab42b719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:31.405039 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cfd6b-89dc-4889-856e-1254ab42b719 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:31.405681 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471cfd6b-89dc-4889-856e-1254ab42b719 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:26:31.408035 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471cfd6b-89dc-4889-856e-1254ab42b719 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:26:31.511059 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1446/2872] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 10 17:26:31 2022] POST /identity/v3/users => generated 290 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 17:26:31.513140 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ef133-4d91-43a3-93ff-0edf8fd6a2e4 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:26:31.525658 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ef133-4d91-43a3-93ff-0edf8fd6a2e4 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:26:31.531015 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe2d4b-76de-4fb7-a471-d12fbf4ef55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:26:31.531672 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe2d4b-76de-4fb7-a471-d12fbf4ef55e admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:31.531927 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe2d4b-76de-4fb7-a471-d12fbf4ef55e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:31.532197 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe2d4b-76de-4fb7-a471-d12fbf4ef55e admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:31.533158 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe2d4b-76de-4fb7-a471-d12fbf4ef55e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:26:31.535081 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe2d4b-76de-4fb7-a471-d12fbf4ef55e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:26:31.553971 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1427/2873] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 17:26:31.560540 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471cfd6b-89dc-4889-856e-1254ab42b719 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:26:31.575198 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471cfd6b-89dc-4889-856e-1254ab42b719 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:26:31.580291 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a7fa1-8e63-46b6-b80c-79d6d99cd3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:26:31.581019 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a7fa1-8e63-46b6-b80c-79d6d99cd3b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:31.581294 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a7fa1-8e63-46b6-b80c-79d6d99cd3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:31.581568 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a7fa1-8e63-46b6-b80c-79d6d99cd3b7 admin admin] PATH_INFO: `/v3/projects/6e9a4bd0385547fca4737fbb5c0ae349/users/2cfa05394d0749ac9ce4516156795f8d/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:31.601792 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22a7fa1-8e63-46b6-b80c-79d6d99cd3b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e9a4bd0385547fca4737fbb5c0ae349, user_id=2cfa05394d0749ac9ce4516156795f8d, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:26:31.604783 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22a7fa1-8e63-46b6-b80c-79d6d99cd3b7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:26:31.607794 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:26:31.680673 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1447/2874] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:26:31 2022] PUT /identity/v3/projects/6e9a4bd0385547fca4737fbb5c0ae349/users/2cfa05394d0749ac9ce4516156795f8d/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:26:31.684733 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe2d4b-76de-4fb7-a471-d12fbf4ef55e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:26:31.695779 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe2d4b-76de-4fb7-a471-d12fbf4ef55e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:26:31.774099 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9933d08-f111-4995-a095-c6a3b52c10d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:26:31.774743 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9933d08-f111-4995-a095-c6a3b52c10d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:31.775016 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9933d08-f111-4995-a095-c6a3b52c10d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:31.775279 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9933d08-f111-4995-a095-c6a3b52c10d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:31.775928 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9933d08-f111-4995-a095-c6a3b52c10d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:26:31.777530 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9933d08-f111-4995-a095-c6a3b52c10d6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:26:31.789165 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1428/2875] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 10 17:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 17:26:31.793590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a7fa1-8e63-46b6-b80c-79d6d99cd3b7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:26:31.805931 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a7fa1-8e63-46b6-b80c-79d6d99cd3b7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:26:31.811109 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a64227-5fb3-4b90-aabf-78f8931016b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:26:31.811787 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a64227-5fb3-4b90-aabf-78f8931016b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:31.812120 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a64227-5fb3-4b90-aabf-78f8931016b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:31.812389 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a64227-5fb3-4b90-aabf-78f8931016b5 admin admin] PATH_INFO: `/v3/projects/6e9a4bd0385547fca4737fbb5c0ae349/users/2cfa05394d0749ac9ce4516156795f8d/roles/7554b353873942fab506de830188aba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:31.814440 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a64227-5fb3-4b90-aabf-78f8931016b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e9a4bd0385547fca4737fbb5c0ae349, user_id=2cfa05394d0749ac9ce4516156795f8d, role_id=7554b353873942fab506de830188aba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:26:31.818289 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a64227-5fb3-4b90-aabf-78f8931016b5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:26:31.831391 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1448/2876] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 10 17:26:31 2022] PUT /identity/v3/projects/6e9a4bd0385547fca4737fbb5c0ae349/users/2cfa05394d0749ac9ce4516156795f8d/roles/7554b353873942fab506de830188aba0 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:26:37.805145 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b3a54-b2fd-4452-a541-ac5269358c95 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:37.806032 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b3a54-b2fd-4452-a541-ac5269358c95 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:37.806509 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b3a54-b2fd-4452-a541-ac5269358c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:37.843615 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f40b3a54-b2fd-4452-a541-ac5269358c95 None None] MFA Rules not processed for user `2cfa05394d0749ac9ce4516156795f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:26:37.844943 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f40b3a54-b2fd-4452-a541-ac5269358c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:26:37.849885 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:26:37.924815 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1429/2877] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:26:37 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:26:37.941153 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a64227-5fb3-4b90-aabf-78f8931016b5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:26:37.943020 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a1a64227-5fb3-4b90-aabf-78f8931016b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:26:38.026281 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a64227-5fb3-4b90-aabf-78f8931016b5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:26:38.032211 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1f8e8-3475-40ad-96ad-74dc05859897 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:26:38.032969 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1f8e8-3475-40ad-96ad-74dc05859897 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:38.033248 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1f8e8-3475-40ad-96ad-74dc05859897 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:38.033516 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1f8e8-3475-40ad-96ad-74dc05859897 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:38.091922 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1f8e8-3475-40ad-96ad-74dc05859897 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:26:38.093502 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1f8e8-3475-40ad-96ad-74dc05859897 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:26:38.103202 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1449/2878] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:26:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:26:41.058976 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087df40-ecd4-4d46-81d9-b143e33cb329 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:41.059700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087df40-ecd4-4d46-81d9-b143e33cb329 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:41.059700 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087df40-ecd4-4d46-81d9-b143e33cb329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:41.080005 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0087df40-ecd4-4d46-81d9-b143e33cb329 None None] MFA Rules not processed for user `a9736ec2b6c54177971db7277ca04c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:26:41.081203 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0087df40-ecd4-4d46-81d9-b143e33cb329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:26:41.091077 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1430/2879] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 10 17:26:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:26:48.098727 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=2,keystone:SELECT=63 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:26:51.074079 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=30 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:26:58.044680 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1f8e8-3475-40ad-96ad-74dc05859897 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:26:58.102651 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1f8e8-3475-40ad-96ad-74dc05859897 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:26:58.106158 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75d5fd-c6f9-462f-85e6-1631cb7a867d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:26:58.106779 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75d5fd-c6f9-462f-85e6-1631cb7a867d service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:58.107052 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75d5fd-c6f9-462f-85e6-1631cb7a867d service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:58.107317 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75d5fd-c6f9-462f-85e6-1631cb7a867d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:58.109060 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3a75d5fd-c6f9-462f-85e6-1631cb7a867d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:26:58.119377 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75d5fd-c6f9-462f-85e6-1631cb7a867d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:26:58.120852 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75d5fd-c6f9-462f-85e6-1631cb7a867d service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:26:58.129755 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1450/2880] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:26:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:26:58.901820 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087df40-ecd4-4d46-81d9-b143e33cb329 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:26:58.901820 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0087df40-ecd4-4d46-81d9-b143e33cb329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:26:58.971036 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087df40-ecd4-4d46-81d9-b143e33cb329 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:26:58.977094 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4180effb-e451-45a9-a25b-95eee4d96bdd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cd39d9a583a4d73a5654017be106019', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:26:58.977706 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180effb-e451-45a9-a25b-95eee4d96bdd service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:26:58.978013 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180effb-e451-45a9-a25b-95eee4d96bdd service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:26:58.978285 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180effb-e451-45a9-a25b-95eee4d96bdd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:26:58.980083 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4180effb-e451-45a9-a25b-95eee4d96bdd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:26:58.992750 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4180effb-e451-45a9-a25b-95eee4d96bdd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:26:58.994325 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4180effb-e451-45a9-a25b-95eee4d96bdd service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:26:59.005098 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1431/2881] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:26:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 17:27:08.125222 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=26 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:27:09.000401 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=32 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:27:14.127732 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75d5fd-c6f9-462f-85e6-1631cb7a867d service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:27:14.130950 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3a75d5fd-c6f9-462f-85e6-1631cb7a867d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:27:14.154512 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75d5fd-c6f9-462f-85e6-1631cb7a867d service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:27:14.162253 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0acd4-7d90-4c12-b2ba-a5e3e67bb125 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:27:14.163007 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0acd4-7d90-4c12-b2ba-a5e3e67bb125 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:27:14.163381 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0acd4-7d90-4c12-b2ba-a5e3e67bb125 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:27:14.163714 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0acd4-7d90-4c12-b2ba-a5e3e67bb125 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:27:14.165897 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-aeb0acd4-7d90-4c12-b2ba-a5e3e67bb125 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:27:14.185839 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0acd4-7d90-4c12-b2ba-a5e3e67bb125 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:27:14.187512 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0acd4-7d90-4c12-b2ba-a5e3e67bb125 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:27:14.202434 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1451/2882] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:27:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:27:16.865413 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4180effb-e451-45a9-a25b-95eee4d96bdd service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:27:16.876064 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4180effb-e451-45a9-a25b-95eee4d96bdd service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:27:16.888425 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db984d69-8661-4a71-8533-bfae5dfff43e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:27:16.889049 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db984d69-8661-4a71-8533-bfae5dfff43e service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:27:16.889348 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db984d69-8661-4a71-8533-bfae5dfff43e service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:27:16.889617 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db984d69-8661-4a71-8533-bfae5dfff43e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:27:16.902475 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db984d69-8661-4a71-8533-bfae5dfff43e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:27:16.904144 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db984d69-8661-4a71-8533-bfae5dfff43e service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:27:16.915397 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1432/2883] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:27:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:27:24.196964 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:27:26.911073 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:28:19.742101 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0acd4-7d90-4c12-b2ba-a5e3e67bb125 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:28:19.743522 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-aeb0acd4-7d90-4c12-b2ba-a5e3e67bb125 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:28:19.758060 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0acd4-7d90-4c12-b2ba-a5e3e67bb125 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:28:19.763362 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123bd234-c8cd-4eca-b998-d1f6bbb3f4a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cd39d9a583a4d73a5654017be106019', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:28:19.763963 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bd234-c8cd-4eca-b998-d1f6bbb3f4a9 service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:28:19.764236 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bd234-c8cd-4eca-b998-d1f6bbb3f4a9 service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:28:19.764503 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bd234-c8cd-4eca-b998-d1f6bbb3f4a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:28:19.766537 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-123bd234-c8cd-4eca-b998-d1f6bbb3f4a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:28:19.779913 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123bd234-c8cd-4eca-b998-d1f6bbb3f4a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:28:19.781473 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123bd234-c8cd-4eca-b998-d1f6bbb3f4a9 service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:28:19.792222 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1452/2884] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:28:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:28:29.787927 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:29:14.109239 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db984d69-8661-4a71-8533-bfae5dfff43e service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:29:14.123909 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db984d69-8661-4a71-8533-bfae5dfff43e service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:29:14.135548 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e65bbe-c1e5-40b1-a608-26576d67ea41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:29:14.135548 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e65bbe-c1e5-40b1-a608-26576d67ea41 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:29:14.135548 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e65bbe-c1e5-40b1-a608-26576d67ea41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:29:14.135548 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e65bbe-c1e5-40b1-a608-26576d67ea41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:29:14.136000 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-77e65bbe-c1e5-40b1-a608-26576d67ea41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:29:14.147901 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e65bbe-c1e5-40b1-a608-26576d67ea41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:29:14.149341 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e65bbe-c1e5-40b1-a608-26576d67ea41 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:29:14.161771 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1433/2885] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:29:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:29:15.347045 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123bd234-c8cd-4eca-b998-d1f6bbb3f4a9 service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:29:15.356412 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123bd234-c8cd-4eca-b998-d1f6bbb3f4a9 service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:29:15.359933 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a08d97-584f-4476-8c06-5faba6b550dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:29:15.360686 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a08d97-584f-4476-8c06-5faba6b550dd service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:29:15.361007 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a08d97-584f-4476-8c06-5faba6b550dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:29:15.361325 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a08d97-584f-4476-8c06-5faba6b550dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:29:15.363057 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-20a08d97-584f-4476-8c06-5faba6b550dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:29:15.376512 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a08d97-584f-4476-8c06-5faba6b550dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:29:15.378196 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a08d97-584f-4476-8c06-5faba6b550dd service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:29:15.389008 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1453/2886] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:29:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:29:24.155937 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:29:25.384167 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:29:27.625335 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e65bbe-c1e5-40b1-a608-26576d67ea41 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:29:27.635287 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e65bbe-c1e5-40b1-a608-26576d67ea41 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:29:27.639103 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3cbbf9-29d2-4585-a861-2b3493a457d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:29:27.639714 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cbbf9-29d2-4585-a861-2b3493a457d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:29:27.639991 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cbbf9-29d2-4585-a861-2b3493a457d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:29:27.640258 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cbbf9-29d2-4585-a861-2b3493a457d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:29:27.642918 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2f3cbbf9-29d2-4585-a861-2b3493a457d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:29:27.653848 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3cbbf9-29d2-4585-a861-2b3493a457d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:29:27.655247 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3cbbf9-29d2-4585-a861-2b3493a457d8 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:29:27.664337 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1434/2887] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:29:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:29:28.079983 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543fff0-5fa6-4ce4-a35d-1717c29433ed None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:29:28.080590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543fff0-5fa6-4ce4-a35d-1717c29433ed None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:29:28.080590 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543fff0-5fa6-4ce4-a35d-1717c29433ed None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:29:28.082646 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1454/2888] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 10 17:29:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:29:28.087867 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7ff21-55af-45ed-9613-c9fedcbad212 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:29:28.088146 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7ff21-55af-45ed-9613-c9fedcbad212 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:29:28.088415 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7ff21-55af-45ed-9613-c9fedcbad212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:29:28.130429 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-00a7ff21-55af-45ed-9613-c9fedcbad212 None None] MFA Rules not processed for user `e1534cfef9e9428bb2e010fff200bc19`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:29:28.131661 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-00a7ff21-55af-45ed-9613-c9fedcbad212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:29:28.141981 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1435/2889] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 10 17:29:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:29:28.153413 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543fff0-5fa6-4ce4-a35d-1717c29433ed None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:29:28.155195 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4543fff0-5fa6-4ce4-a35d-1717c29433ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:29:28.169501 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543fff0-5fa6-4ce4-a35d-1717c29433ed None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:29:28.175983 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caff11e-9fbf-45de-b3c8-d2a93510b41d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:29:28.176593 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caff11e-9fbf-45de-b3c8-d2a93510b41d service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:29:28.176869 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caff11e-9fbf-45de-b3c8-d2a93510b41d service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:29:28.177139 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caff11e-9fbf-45de-b3c8-d2a93510b41d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:29:28.184890 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caff11e-9fbf-45de-b3c8-d2a93510b41d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:29:28.186337 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caff11e-9fbf-45de-b3c8-d2a93510b41d service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:29:28.194699 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1455/2890] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:29:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:29:28.289895 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4763051-4ec9-4245-b138-f7ef9be8bccf None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:29:28.290172 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4763051-4ec9-4245-b138-f7ef9be8bccf None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:29:28.290479 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4763051-4ec9-4245-b138-f7ef9be8bccf None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:29:28.291890 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1436/2891] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 10 17:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 17:29:28.297473 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642eb38-7b6f-4a69-b984-b5a2c1bc996b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:29:28.298282 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642eb38-7b6f-4a69-b984-b5a2c1bc996b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:29:28.298586 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642eb38-7b6f-4a69-b984-b5a2c1bc996b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:29:28.336006 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f642eb38-7b6f-4a69-b984-b5a2c1bc996b None None] MFA Rules not processed for user `615cc36e21cc493b823e6cee695ac308`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 17:29:28.337256 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f642eb38-7b6f-4a69-b984-b5a2c1bc996b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:29:28.347239 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1456/2892] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 10 17:29:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 17:29:28.358126 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4763051-4ec9-4245-b138-f7ef9be8bccf None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:29:28.370138 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4763051-4ec9-4245-b138-f7ef9be8bccf None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:29:28.373765 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20991aec-3914-4257-8282-add664ebded3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:29:28.374373 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20991aec-3914-4257-8282-add664ebded3 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:29:28.374658 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20991aec-3914-4257-8282-add664ebded3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:29:28.374925 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20991aec-3914-4257-8282-add664ebded3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:29:28.386397 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20991aec-3914-4257-8282-add664ebded3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:29:28.387827 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20991aec-3914-4257-8282-add664ebded3 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:29:28.399169 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1437/2893] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:29:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:29:38.333193 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=15 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:29:38.394506 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=21 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:30:20.394022 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f642eb38-7b6f-4a69-b984-b5a2c1bc996b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:30:20.403937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f642eb38-7b6f-4a69-b984-b5a2c1bc996b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:30:20.408598 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f2f34-431c-4a89-a390-7dcc1e534df6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1534cfef9e9428bb2e010fff200bc19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:30:20.409224 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f2f34-431c-4a89-a390-7dcc1e534df6 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:30:20.409505 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f2f34-431c-4a89-a390-7dcc1e534df6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:30:20.409773 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f2f34-431c-4a89-a390-7dcc1e534df6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:30:20.411656 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0c9f2f34-431c-4a89-a390-7dcc1e534df6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:30:20.422804 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9f2f34-431c-4a89-a390-7dcc1e534df6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:30:20.424259 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9f2f34-431c-4a89-a390-7dcc1e534df6 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:30:20.433206 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1457/2894] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:30:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:30:30.429128 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:31:01.371413 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20991aec-3914-4257-8282-add664ebded3 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:31:01.386873 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20991aec-3914-4257-8282-add664ebded3 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:31:01.393630 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87308bd-986b-4a4f-8e64-2f04fcfcb585 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '615cc36e21cc493b823e6cee695ac308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:31:01.394249 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87308bd-986b-4a4f-8e64-2f04fcfcb585 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:31:01.394552 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87308bd-986b-4a4f-8e64-2f04fcfcb585 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:31:01.394822 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87308bd-986b-4a4f-8e64-2f04fcfcb585 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:31:01.397028 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a87308bd-986b-4a4f-8e64-2f04fcfcb585 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:31:01.411998 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87308bd-986b-4a4f-8e64-2f04fcfcb585 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:31:01.413566 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87308bd-986b-4a4f-8e64-2f04fcfcb585 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:31:01.432770 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1438/2895] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:31:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:31:11.423708 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:31:11.460774 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f2f34-431c-4a89-a390-7dcc1e534df6 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:31:11.474276 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f2f34-431c-4a89-a390-7dcc1e534df6 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:31:11.479310 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8cad3-3ae6-4a98-9783-4b44fd6c05bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cd39d9a583a4d73a5654017be106019', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:31:11.479914 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8cad3-3ae6-4a98-9783-4b44fd6c05bd service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:31:11.480196 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8cad3-3ae6-4a98-9783-4b44fd6c05bd service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:31:11.480498 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8cad3-3ae6-4a98-9783-4b44fd6c05bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:31:11.482254 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-deb8cad3-3ae6-4a98-9783-4b44fd6c05bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:31:11.494838 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8cad3-3ae6-4a98-9783-4b44fd6c05bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:31:11.496458 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8cad3-3ae6-4a98-9783-4b44fd6c05bd service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:31:11.507664 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1458/2896] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:31:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:31:15.129053 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87308bd-986b-4a4f-8e64-2f04fcfcb585 service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:31:15.199139 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87308bd-986b-4a4f-8e64-2f04fcfcb585 service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:31:15.204401 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2bdec1-8500-466a-8ce8-f21d23cf6231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:31:15.205028 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bdec1-8500-466a-8ce8-f21d23cf6231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:31:15.205312 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bdec1-8500-466a-8ce8-f21d23cf6231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:31:15.205582 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bdec1-8500-466a-8ce8-f21d23cf6231 admin admin] PATH_INFO: `/v3/users/a9736ec2b6c54177971db7277ca04c1d` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:31:15.206760 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2bdec1-8500-466a-8ce8-f21d23cf6231 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9736ec2b6c54177971db7277ca04c1d)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:31:15.208821 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2bdec1-8500-466a-8ce8-f21d23cf6231 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:31:15.368415 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae2bdec1-8500-466a-8ce8-f21d23cf6231 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9736ec2b6c54177971db7277ca04c1d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:31:15.433896 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae2bdec1-8500-466a-8ce8-f21d23cf6231 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9736ec2b6c54177971db7277ca04c1d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:31:15.447563 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae2bdec1-8500-466a-8ce8-f21d23cf6231 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9736ec2b6c54177971db7277ca04c1d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:31:15.448254 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae2bdec1-8500-466a-8ce8-f21d23cf6231 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9736ec2b6c54177971db7277ca04c1d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:31:15.448889 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae2bdec1-8500-466a-8ce8-f21d23cf6231 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9736ec2b6c54177971db7277ca04c1d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:31:15.463801 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1439/2897] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:31:15 2022] DELETE /identity/v3/users/a9736ec2b6c54177971db7277ca04c1d => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:31:15.727735 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8cad3-3ae6-4a98-9783-4b44fd6c05bd service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:31:15.739642 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8cad3-3ae6-4a98-9783-4b44fd6c05bd service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:31:15.798203 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9825f-da37-4a1d-998c-f3aac1a89b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:31:15.798871 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9825f-da37-4a1d-998c-f3aac1a89b42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:31:15.799141 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9825f-da37-4a1d-998c-f3aac1a89b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:31:15.799404 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9825f-da37-4a1d-998c-f3aac1a89b42 admin admin] PATH_INFO: `/v3/projects/c1f04e9a75b44271a9a2aa3834c5c9d4` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:31:15.800527 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f9825f-da37-4a1d-998c-f3aac1a89b42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1f04e9a75b44271a9a2aa3834c5c9d4)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:31:15.802441 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f9825f-da37-4a1d-998c-f3aac1a89b42 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:31:15.904047 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a8f9825f-da37-4a1d-998c-f3aac1a89b42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1f04e9a75b44271a9a2aa3834c5c9d4', 'request_id': 'req-a8f9825f-da37-4a1d-998c-f3aac1a89b42'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:31:15.911127 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a8f9825f-da37-4a1d-998c-f3aac1a89b42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1f04e9a75b44271a9a2aa3834c5c9d4', 'request_id': 'req-a8f9825f-da37-4a1d-998c-f3aac1a89b42'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:31:15.970882 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a8f9825f-da37-4a1d-998c-f3aac1a89b42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1f04e9a75b44271a9a2aa3834c5c9d4', 'request_id': 'req-a8f9825f-da37-4a1d-998c-f3aac1a89b42'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:31:15.982419 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a8f9825f-da37-4a1d-998c-f3aac1a89b42 admin admin] The token cache is being invalidate because project c1f04e9a75b44271a9a2aa3834c5c9d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:31:15.982709 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a8f9825f-da37-4a1d-998c-f3aac1a89b42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:31:15.983393 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a8f9825f-da37-4a1d-998c-f3aac1a89b42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:31:15.984958 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1459/2898] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:31:15 2022] DELETE /identity/v3/projects/c1f04e9a75b44271a9a2aa3834c5c9d4 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:31:15.989190 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2bdec1-8500-466a-8ce8-f21d23cf6231 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:31:15.990959 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ae2bdec1-8500-466a-8ce8-f21d23cf6231 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:31:16.005503 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2bdec1-8500-466a-8ce8-f21d23cf6231 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:31:16.064769 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a400b-67da-426c-a6b0-ed960d5fd6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:31:16.069510 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a400b-67da-426c-a6b0-ed960d5fd6ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:31:16.069510 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a400b-67da-426c-a6b0-ed960d5fd6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:31:16.069510 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a400b-67da-426c-a6b0-ed960d5fd6ab admin admin] PATH_INFO: `/v3/users/2cfa05394d0749ac9ce4516156795f8d` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:31:16.069510 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a400b-67da-426c-a6b0-ed960d5fd6ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cfa05394d0749ac9ce4516156795f8d)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:31:16.069510 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a400b-67da-426c-a6b0-ed960d5fd6ab admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:31:16.091062 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:31:16.225985 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b79a400b-67da-426c-a6b0-ed960d5fd6ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cfa05394d0749ac9ce4516156795f8d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:31:16.268091 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b79a400b-67da-426c-a6b0-ed960d5fd6ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cfa05394d0749ac9ce4516156795f8d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:31:16.281673 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b79a400b-67da-426c-a6b0-ed960d5fd6ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cfa05394d0749ac9ce4516156795f8d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:31:16.282390 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b79a400b-67da-426c-a6b0-ed960d5fd6ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cfa05394d0749ac9ce4516156795f8d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:31:16.283012 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b79a400b-67da-426c-a6b0-ed960d5fd6ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cfa05394d0749ac9ce4516156795f8d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:31:16.296090 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1440/2899] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 10 17:31:15 2022] DELETE /identity/v3/users/2cfa05394d0749ac9ce4516156795f8d => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:31:16.551705 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9825f-da37-4a1d-998c-f3aac1a89b42 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:31:16.553610 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a8f9825f-da37-4a1d-998c-f3aac1a89b42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:31:16.572196 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9825f-da37-4a1d-998c-f3aac1a89b42 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:31:16.638937 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94553d6a-278f-46db-8c77-dcb34a581b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164faaac1ec04c50ba5358bf58d221cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aff7d0faeed34f6193c691567931a8ff', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:31:16.640480 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94553d6a-278f-46db-8c77-dcb34a581b1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:31:16.640675 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94553d6a-278f-46db-8c77-dcb34a581b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:31:16.641067 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94553d6a-278f-46db-8c77-dcb34a581b1d admin admin] PATH_INFO: `/v3/projects/6e9a4bd0385547fca4737fbb5c0ae349` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:31:16.642099 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94553d6a-278f-46db-8c77-dcb34a581b1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e9a4bd0385547fca4737fbb5c0ae349)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:31:16.644051 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94553d6a-278f-46db-8c77-dcb34a581b1d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:31:16.665559 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:31:16.760671 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-94553d6a-278f-46db-8c77-dcb34a581b1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e9a4bd0385547fca4737fbb5c0ae349', 'request_id': 'req-94553d6a-278f-46db-8c77-dcb34a581b1d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:31:16.767518 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-94553d6a-278f-46db-8c77-dcb34a581b1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e9a4bd0385547fca4737fbb5c0ae349', 'request_id': 'req-94553d6a-278f-46db-8c77-dcb34a581b1d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:31:16.851497 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-94553d6a-278f-46db-8c77-dcb34a581b1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e9a4bd0385547fca4737fbb5c0ae349', 'request_id': 'req-94553d6a-278f-46db-8c77-dcb34a581b1d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:31:16.863838 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-94553d6a-278f-46db-8c77-dcb34a581b1d admin admin] The token cache is being invalidate because project 6e9a4bd0385547fca4737fbb5c0ae349 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 17:31:16.864128 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-94553d6a-278f-46db-8c77-dcb34a581b1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:31:16.864838 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-94553d6a-278f-46db-8c77-dcb34a581b1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 17:31:16.866330 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1460/2900] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 10 17:31:16 2022] DELETE /identity/v3/projects/6e9a4bd0385547fca4737fbb5c0ae349 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 17:31:26.291984 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:31:26.868175 dv-cinder-852724-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 17:33:23.679873 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a400b-67da-426c-a6b0-ed960d5fd6ab admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 17:33:23.681308 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b79a400b-67da-426c-a6b0-ed960d5fd6ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:33:23.823241 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a400b-67da-426c-a6b0-ed960d5fd6ab admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 17:33:23.892842 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bae13c-c41d-4beb-8eff-853e3ea53a46 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cd39d9a583a4d73a5654017be106019', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd11f33af53374dffa61132642a7770f4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 17:33:23.892842 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bae13c-c41d-4beb-8eff-853e3ea53a46 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 17:33:23.892842 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bae13c-c41d-4beb-8eff-853e3ea53a46 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 17:33:23.892842 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bae13c-c41d-4beb-8eff-853e3ea53a46 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 17:33:23.893491 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-54bae13c-c41d-4beb-8eff-853e3ea53a46 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 17:33:23.898078 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bae13c-c41d-4beb-8eff-853e3ea53a46 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 17:33:23.899813 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bae13c-c41d-4beb-8eff-853e3ea53a46 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 17:33:23.909936 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1441/2901] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 10 17:33:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 17:33:33.905646 dv-cinder-852724-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}